Dagster Labs

Thanks to everyone who joined us for the Dagster+ launch! Dagster+ is an exciting redefinition of #dataorchestration aimed at: • reducing spend via cost #observability • eliminating error-prone point-solutions • enabling #data teams to operate autonomously Learn more about Dagster+ and get started for free here: https://bit.ly/3JnpZ0l

One of my favorite parts of working at Dagster is getting inside access to some of the best engineers I've had the pleasure of working with. Here's a great example of an RFC by Owen Kephart on the upcoming Declarative Automation work we are building, I learn a lot just by reading how Owen both illustrates problems and communicates his approach. https://lnkd.in/gA-ag6Ve

Archiving is one of the last (and most critical) stages of a #data lifecycle management strategy, when data that is no longer actively used is sent to a separate storage system for long-term retention. This can: • optimize system performance • reduce primary storage costs • ensure compliance Explore different data archiving strategies here: https://bit.ly/3zscYRx

On Episode 43 of the Podcast Ruined by a Software Engineer, Sandy Ryza joins Perry Tiu and discusses: • transitioning to and from software engineering and data science • data application performance • maintaining vast open-source projects Hear Sandy's insights on these (and more) and catch the full podcast here: https://bit.ly/4clow7V Alternatively, if you want to watch it, go here: https://bit.ly/4crAT2i

We are looking for our first IT & Security Lead here are Dagster Labs. Please DM me if you have any questions or interested in applying!

In Ep. 39 of the Value Driven Data Science podcast, Sandy Ryza joins Dr. Genevieve Hayes to discuss data science’s role in data orchestration, AI, ML, and next-gen tools. He also shares insights on how his time in data science has affected his engineering at Dagster. Check it out here: https://bit.ly/4bgvSbt ---------- Follow Dagster Labs to learn about #dataengineering and building a productive #data platform through our talks, deep dives, and #Dagster updates.

Data clustering is the grouping of data points in such a way that points in the same group (or cluster) are more similar to each other than to those in other groups. Clustering helps in identifying patterns and anomalies, enhancing the efficiency of predictive models, and improving the accuracy of machine learning algorithms. By categorizing data into clusters, data engineers and analysts can streamline processes, tailor strategies to specific groups, and achieve more precise outcomes. Explore data clustering with a Python example here: https://bit.ly/3Xv4cMO ---------- Follow Dagster Labs to learn about #dataengineering and building a productive #data platform through our talks, deep dives, and #Dagster updates.

Just published my latest article on using Dagster for an Incremental ETL Pipeline! As data engineers, managing ETL processes efficiently, especially for incremental data extraction, is a constant challenge. In this article, I delve into how I leveraged Dagster’s resources, assets, jobs, and schedules to build an incremental ETL pipeline. Check it out and let me know your thoughts! 👉 Read the article here: https://lnkd.in/eW9hEnaN #DataEngineering #ETL #Dagster #DataPipeline

Demos, swag, raffles, and our smiling faces-- a few things you have likely encountered if you've run into us at the Databricks #DataAISummit. There's still time to see us and talk #dataengineering! We're at Booth 36! Also, congrats to Anthony (from Robinhood) on winning a Nintendo Switch!