Kaivalya Apte

We’ve reached a point where managing digital cookies burns more calories than you’d gain from eating real ones.

Such a pain 😵‍💫

If I ever design a system badly…

I won’t admit..

But there will be signs.

- Every software engineer ever.

Haha I have the same story.. so much thinking, too little writing 😅

Planning a write up?

I never thought about usage of my face and voice for training tbh. And I don’t think I can control that too much.

Part of the job I guess.

Thanks @qianli.dev

Challenges faced:

- Remote recording is sometimes challenging, but these days there are many tools like Riverside, Descript make it really easy.

- Not getting demotivated is a challenge. But ultimately you start enjoying the process and engagement becomes a side effect.

1/12. I'm excited to share our latest technical blog post on ParadeDB.

After a brief hiatus focused on transforming ParadeDB into an enterprise-ready database, expect to hear a lot more from us.

Today's post: How ParadeDB built an LSM on top of Postgres block storage. 🧵

LSM Trees: How does it all work? YouTube video by The Geek Narrator

How do LSM trees work?

Finally created a light board video. I hope you like it 🙏🏻

LSM Trees: How does it all work?
youtu.be/CizxYuq-z6E

👏 thanks

Agreed. Payments are really complicated and defining the states in a simple post like this is impossible. 😅 it was totally a simplistic model to share the idea of thinking in terms of state.

Absolutely. Cannot agree more

Another advantage to designing systems as state machines: automated testing and even verification becomes dramatically easier.

You're not just writing code. You're orchestrating state transitions at every level - aggregating them, filtering them, transforming them into something users can consume.

Follow @thegeeknarrator.bsky.social if you are interested in databases, distributed systems and software engineering tips.

Everything is a state machine - from Netflix's video player (LOADING → PLAYING → PAUSED → ERROR) to your payment processing (PENDING → AUTHORIZED → SETTLED → FAILED).

Even your individual functions have states. Even the bits in memory have states (0 or 1).

Then, do the following and find more states:

Think about what happens when things fail
Think about user actions that can happen simultaneously
Think about external events and timeouts

Modelling your system explicitly is the bottom line.

How do you know you are modelling the right states?

Try to find:
"boolean flags scattered everywhere"
"nested if-else chains"
"this should never happen" comments
"works-on-my-machine" bugs
"race-condition" issues
"impossible-to-reproduce" errors

It can help you create a simple model of your system that you can prototype and test.

In my experience it brings predictable behaviour and keeps away impossible system states.

What is State Machine thinking and how do I use it (practically)?

State machine thinking is one of the best ways to build reliable software. The idea is to break down complex systems into discrete states and explicit transitions between them.

When you do this, in my experience you pretty much end up being on the right track and make good progress.

But it will take practice and you will find yourself stuck, which is completely normal.

Then, do the following and find more questions:

- Thinking about evidences or ways of testing for your findings
- Think about alternative perspectives
- Think about consequences and implications

How do you know you are asking the right questions?

Try to find
- “assumptions”
- “stereotypes”
- “dogmas”
- “because-someone-said-so”,
- “because-its-written-in-a-book”
- “its-obvious”

kinda statements and challenge them. When you try to challenge them you pretty much ask the right questions.

It takes a lot of thinking and practice and may also feel unproductive at the beginning (although it’s fun).

In my experience it brings creative solutions and keeps away unnecessary complexity.

Asking the right and enough questions is the bottom line.

What is First principles thinking and how do I use it (practically) ?👇

First-principles thinking is one of the best ways to tackle complex problems.

The idea is to break down complex problems into primitive elements and then reassemble the problem from scratch.

I have had an even better experience with Gemini though. Did you try that?

Small enough service is a micro-service.

Large enough language model is a Large Language Model.

Naming is a NP-hard problem.

Here is a sketch of how a solid observability platform built on top of object storage would look like this 👇

And guess what, Parseable is built exactly like that. All the good stuff that makes a system fast, reliable and highly scalable.

New episode 🚀
youtu.be/MugLlrf5E_g

Diskless Kafka?

But why?

Why is it important for Kafka ecosystem ?

How does it compare with existing revolutionary technologies?

Blogged a comparison between Diskless Kafka, WarpStream and Confluent Freight. www.geeknarrator.com/blog/diskles...

Diskless Kafka?

But why?

Why is it important for Kafka ecosystem ?

How does it compare with existing revolutionary technologies?

Blogged a comparison between Diskless Kafka, WarpStream and Confluent Freight. www.geeknarrator.com/blog/diskles...

Blogged ✍🏻

Schema Driven Development with Buf

www.geeknarrator.com/blog/buf-sch...

DUPLICATE events and painfully SLOW joins are a real and painful problems in the world of stream processing and realtime analytics.

But why there hasn't been a simple, easy and fast solution for these problems that just works with the hassle?