Last friday I published my most personal Substack post so far. I tell the tale of my job hopping in Data Engineering early in my career.
Check it out:
thedatasitter.substack.com/p/my-history...
Let's talk about Partitioning and Bucketing as a performance strategy for Spark!
open.substack.com/pub/thedatas...
Friday's post will be about the Architect role. Hope you guys like it!
After some good years of job hopping, today is the first time I have a one-year anniversary at a company. Gotta say, I love my job at Caylent
How can AI kill software engineering jobs if it can't even kill SQL? lol
How do you guarantee you won't have any bad rows maculating your source of truth?
Your answer is here:
thedatasitter.substack.com/p/netflixs-w...
Nevertheless, the spirit of the division is the same. You'll have your go-to production data at the end - the gold or analytics layer.
These are supposed to be the most trusted, up-to-date, clean, and good looking tables.
AWS names them “raw, staging, and analytics.”
Databricks likes the medallion convention of “bronze, silver, and gold.”
In some scenarios, you would add a layer for events. Some call them “transient,” others “landing.”
If you hold 10 data engineers in a room and ask them the correct naming convention for the layers of a data lake, odds are you'll receive 11 different answers.
Also, if you missed, this week I wrote about parquet files and vectors. You can take a look at thedatasitter.com
Just finished preparing The Data Sitter's Friday post. I'll discuss memoir books. Hope it doesn't feel off, as it is a tech blog
“My kid never got vaccinated and he’s fine”
The kid:
The most “going to gym in a small brazilian town” pic you’re going to see today:
preach
So far, this has been the book of the year for me. I had no idea of how messy is Elon’s management of Twitter/X.
There was a time when he ended contracts with the companies that cleaned the Twitter offices lol
Employees had to bring toilet paper from home or go to bathrooms in nearby coffee shops
The only thing that can pacify the world nowadays
I tried and failed to study math for ML for 3 years now, and finally now, after treatment started, I can concentrate on these complex topics and actually learn lol
Earlier this year I've been diagnosed with ADHD. I started medication on Monday w/ Concerta and it's been life changing.
What's the best `hello_world` for Bluesky?
