If you recall this project github.com/jeremycole/i... to parse & research InnoDB structures, we now have another one in Rust for MariaDB by surprise github.com/sitano/mdbutil. Unfinished, but works.
How could it be then you would ask? The answer is simple - redo log is in different directory and the SST script does not account for that. That boring after all that hardcore stuff.
Then easy-peazy lemon-squizy using logN manual binary-search with my own hands and LLDB I have arrived at SST during LSN was updating. The only thing there that does that is data transfer. It turned out it actually is data transfer. With the only difference - redo log is not t-ed
Since I could test system tablespace state I have found out with the help of LLDB that the RSEG LSN changing even before Redo Log initialization (recovery) process on boot. That should mean that it is tampered even before InnoDB. The only serious thing that is there bef is WSREP.
So when and how page LSN may be modified? Usually, they are modified during execution IIANW (lol whatsthat) of mini-transactions commit if there were any modifications. Just grep for FIL_PAGE_LSN and mach writes in the code. By using LLDB I now could see non of that points parted
What is page LSN though? It is the last record (MTR actually) modification logical sequence number that is there to track page version. It is used during recovery or page load mechanism to verify data integrity. Page checksum and all necessary metadata is also there.
in bash this `printf "%d" 0x000144dc` this gives you 83164 u64 that is actually the page's LSN. 1st thing to mention here is that any tablespace page has a header (and a footer). In MDB the page header is 38 bytes (see FIL_PAGE_DATA). Page LSN is at offset 16 (FIL_PAGE_LSN).
It is initialized in trx_rseg_array_init() at start during InnoDB engine init in trx_lists_init_at_db_start() after Redo Log has been looked through. With that in mind I was able to peek the changes to page 6 by just looking at 0x18000 with xxd / grep: 6146:00018018: 0001 44dc
System tablespace in MariaDB stored in ib_data1 and all 16KB pages are aligned. That means 6th page is at 16KB*6 = 98304 in ib_data1 or at 0x18000 physical offset. System tablespace has at least 8 known lists defined at fsp0types.h one of which is the page 6 rollback segment head
Recently saw weird bugs when rollback segments versions ran away from redo log. It turned out its an SST bug :D However, what I did to debug it? I took backtrace of the error message and tracked it to the system tablespace page #6. It turned out ths page is a trx rseg list head.
In a full clean (slow) shutdown, the last entry in the log will always be a FILE_CHECKPOINT that bears data LSN for InnoDB tablespaces. It is accepted if CheckpointLsnEntry == EndOfTheLog and the log actually has it at this pos. Obv., you don't need redo log to start if its good.
If you ever thought what Redo log may look like, here you go: it is chains of micro-edits:
MTR Chain count=4, len=27, lsn=11344769
1: Mtr { space_id: 4, page_no: 3, op: Memset }
2: Mtr { space_id: 4, page_no: 3, op: Write }
3: Mtr { space_id: 4, page_no: 3, op: Memset }
...
If you get bored, I looked through the MariaDB redo log structure to understand what it is made of. Here is the notes sitano.github.io/mariadb/inno... and there is MTR reader in Rust with lots of TODO that can generate fake checkpoints. github.com/sitano/mdbutil
Got it. Worth adding this elaboration to the text.
What is “keying time”? Open/close loop that goes further should be about coordinated omission?
I have just ported ngram plugin to MariaDB to allow normal contains() requests with all the new tests only to find out it was done by someone 4 years ago… what a pity. At least I have learned what a rabbit hole this is. Especially for CJK.
I just saw a very weird issue with shared dlopened library that gets at some point reloaded by something without applying relocation table. No GDB awatches on its rw-p mem addresses catch it, nor any of dlopen, dlclose, _dl_* or mmap functions. Confused.
I’m constantly amazed by the historical practice of writing C software in a way that results in source files reaching 20,000 lines of code each.
All knows vacuum problems is a myth, ahahahahahah)
Just thought what if Nvidia actually can't make anymore hardware improvements to their GPUs by not just putting more cores into the chip and considering they have almost achieved power and thermal limit with 4xxx-5xxx they have no choice but sell software features. But even that will end soon.
Skepticism I share too: The Generative AI Con by Edward Zitron www.wheresyoured.at/longcon/
The problem is they are selling it like a next-gen performance improvement while it is not. New cards “almost” even weaker in normal rendering.
Exactly. Even with a single frame frame gen it feels very bad. Maybe its a game problem but nothing will fix input lag.
Frame gen is the biggest BS out there. Like 20 frames to 250.
Want to leave it for the future - Nvidia is a modern evil company that is scamming their users (gaming sector) with marketing-lies and high prices at least since the era of mining boom. So disappointing.
Big problem with seniority is that by the time and age you (me) get there its time to become an expert in something. Try to cold proof it on the market without a history of related experience.
Redis Labs OSS clients raid is miraculous nonsense github.com/redis-rs/red....
It looks like everything is cheaper than datadog :D
How to get started in Graphics Programming in 2024: what you need to know + my favorite resources
