TEGNicholas.bsky.social

Matt Yglesias Is Confidently Wrong About Everything The Biden administration’s favorite centrist pundit produces smug pseudo-analysis that cannot be considered serious thought. He ought to be permanently disregarded.

"[Matt Yglesias] can write whole essays claiming that fracking is good and we need fossil fuel friendly energy policies, dismissing progressives as childish, while never engaging with the scientific literature on the consequences of climate change"

This reads like the writings of climate denialists - constant strawmanning, shifting of goalposts, selection bias in research quoted, and conspiracy theorizing.

We Needed Better Cloud Storage for Python so We Built Obstore — Development Seed Obstore solves the friction we kept hitting in cloud-native workflows.

New blog post on Obstore, fast, multi-provider cloud storage access for Python:

developmentseed.org/blog/2025-08...

Open source, open science for earth, climate and geospatial science? Coming to #AGU25? Build tools in #Python @jupyter.org?

Submit an abstract for this session and come meet us and like minded scientists!

He helped Microsoft build AI to help the climate. Then Microsoft sold it to Big Oil. A former Microsoft project manager reveals how the tech giant is using AI to help Big Oil drill—and how he and his partner are now pushing for change.

Will Alpine co-wrote Microsoft's manifesto on how AI will be a powerful force for good for climate change

In a new interview, Alpine disavows the manifesto, saying he believes Microsoft used his work to distract from the much larger climate harms the company enables through contracts with Big Oil

This is fucking insane. Closing these NOAA labs would obliterate our ability to observe, understand, and forecast the Earth System, from weather systems tomorrow to sea levels 50 years from now.

NASA is being told to cancel 19 *active* missions to save $6B, which looks to be less than the ICE *hiring/retention* budget going forward.

I need people to let that sentence sink into their bones for a minute.

I've been adding new accounts to the Open Source Geospatial starter pack. Who else wants on or off?
#gischat #geosky

go.bsky.app/PGYLmPG

Oh @jsignell.github.io too!

I would have suggested Max Jones, Aimee Barciauskas, or Lindsey Nield, but they don't seem to be on BlueSky, so you could instead add @jhamman.bsky.social , @rabernat.bsky.social , or myself.

Zarr takes Cloud-Native Geospatial by storm - Earthmover Our takeaways from the Cloud-Native Geospatial conference on Zarr’s surging adoption and its impact on the future of Earth Observation data. Our team just returned from an action-packed week at the Cl...

There should be some @zarr.dev geospatial representation on here.

Evidence:

earthmover.io/blog/zarr-ta...

It's outrageous that NASA GISS, one of the best earth & space science labs in the world, is being kicked out of its Columbia home. The outstanding scientists who work there can't say that publicly, but I can. And so can you --- call your reps, esp. (but not only) if you live in NYC or NY state.

𝐻𝑜𝑤 𝑑𝑜𝑒𝑠 𝐼𝑐𝑒𝑐ℎ𝑢𝑛𝑘 𝑎𝑣𝑜𝑖𝑑 𝑟𝑒𝑑𝑢𝑛𝑑𝑎𝑛𝑡 𝑠𝑡𝑜𝑟𝑎𝑔𝑒 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑑𝑎𝑡𝑎 𝑣𝑒𝑟𝑠𝑖𝑜𝑛𝑠?

Icechunk stores only new or changed chunks for each version —no redundant copies or rewrites. You get instant time travel, branching, and efficient updates, all with negligible storage overhead.

More: bit.ly/3F1XFST

Excellent post by Brian Davis laying out why doing "Open Science" for data-driven workflows is almost impossible in practice, at least without much better data pipeline tools.

nice analogy 😉

White House Proposal Could Gut Climate Modeling the World Depends On Potential funding cuts for NOAA and its research partners threaten irreparable harm not only to climate research but to American safety, competitiveness, and national security.

The proposed cuts to NOAA cold have profound consequences not just for climate change, but for our national security and the entire economy. Here's what I learned: www.propublica.org/article/trum...

It's fun to work with real hardcore software engineers like @functionth.bsky.social who can teach you about database consistency and transactions and all that

Scientific data infrastructure should be built on solid foundations like this instead of on piles of janky code written by postdocs...

the fact that I've never once thought about making a range request, and yet make them constantly for extremely targeted data pulls, is absolutely an invisible technical miracle

Of course- I really wrote this article for my past self! I wish someone had explained this cloud science stuff to me earlier.

It’s also important for understanding what problem VirtualiZarr solves.

I’ve given this explanation to many people in the past (including at @cworthy.bsky.social), so I hope that this article can serve as a useful reference the next time someone wonders what @zarr.dev actually is.

I wrote the article I wish I could have read back when I first heard of Zarr and cloud-native science back in 2018.

This explains how object storage and conventional filesystems are different, and the key properties that make @zarr.dev work so well in cloud object storage.

5/ Almost all organisations working with scientific array data have this kind of data delivery issue, even if it's just internally.

Whilst the Flux integrations today are established geospatial standards, you also see similar patterns in other fields such as Neuroscience.

4/ Flux's architecture is auto-scaling, so once turned on there is no need to worry about how many users are hitting the data.

As it's not a stateful server like THREDDS, it won't catch fire under pressure.

This is what "Cloud-Native" architectures for scientific data look like.

3/ Your downstream scientists, GIS users, analysts, and external users can all now forget about file formats!

They just keep using the same GUI or tool or script that they prefer, and don't need any other services or copies of the data made bespoke for them - Flux does that on-demand!

2/ Flux bridges this chasm.

It sits in between your data and the consumers, springing up at a moment's notice to provide subsets of data however your users prefer it.

1/ Flux solves the impedance mismatch between geospatial data providers and consumers.

Providers want to manage data lakes stored in cloud-optimized formats like Zarr, but consumers want their applications to keep being fed data in ways they already understand.

White House outlines plan to gut NOAA, smother climate research The agency’s Office of Oceanic and Atmospheric Research would be “eliminated as a line office,” according to a memo from the Office of Management and Budget.

Hard to overstate this plan's reach, which touches nearly every aspect of NOAA's work - dissolving its research arm, gutting climate science, diminishing sat observations, boosting fossil fuels. With amazing colleagues Daniel Cusick and @scottpwaldman.bsky.social :
www.politico.com/news/2025/04...

0$ Data Distribution Ju Data Engineering Weekly - Ep 78

You could also do this for arbitrarily large scientific array datasets using Xarray + Icechunk + R2/Tigris

juhache.substack.com/p/0-data-dis...

Exploring Icechunk scalability: untangling S3's prefix story | Earthmover We show Icechunk can scale to extremely high concurrency levels, and explain how it achieves this in modern object stores.

📣 Blog post alert! 𝐄𝐱𝐩𝐥𝐨𝐫𝐢𝐧𝐠 𝐈𝐜𝐞𝐜𝐡𝐮𝐧𝐤 𝐬𝐜𝐚𝐥𝐚𝐛𝐢𝐥𝐢𝐭𝐲: 𝐮𝐧𝐭𝐚𝐧𝐠𝐥𝐢𝐧𝐠 𝐒𝟑'𝐬 𝐩𝐫𝐞𝐟𝐢𝐱 𝐬𝐭𝐨𝐫𝐲. This technical post by @functionth.bsky.social dives deep into the internals of how S3 shards data, showing that distributed Icechunk can easily perform 230,000 object reads/sec and beyond. earthmover.io/blog/explori...

Several times some database comp sci nerd has suggested to me that you could just do everything in array land using tabular database tools. Whilst they are technically correct that you _could_, this article convincingly shows why you _should not_ - that would be horribly inefficient.