Most “TikTok analytics” tools were never built for TikTok

I build a TikTok analytics product, so take this with the appropriate grain of salt. But after shipping busypipe and studying the tools people line it up against — Exolyt, Pentos, Analisa.io, Iconosquare — I keep landing on the same conclusion: the category has two problems working against you at the same time. One is technical. One is about money. And both trace back to a single fact — most of the software sold to help you understand TikTok was designed for a different platform, in a different era, for a different customer.

Once you see it, you can’t unsee it.

Problem one: they’re Instagram tools wearing a TikTok costume

TikTok does not work like Instagram, and it isn’t close.

On Instagram, follower count is destiny. Reach roughly tracks audience size, and a snapshot of a profile — followers, average likes, top posts — tells you most of what you need to know. So the analytics tools that grew up on Instagram are built around exactly that: you search a profile, you get a snapshot, you move on.

TikTok breaks that model. Reach is earned per video, not granted by follower count — a brand-new account can go viral, and a huge one can flop. What actually decides your fate is momentum: which sounds, formats, and hashtags are accelerating right now, and how fast a creator is climbing. That’s a rate of change, and you cannot see a rate of change in a one-time snapshot. You can only see it by watching over time.

This is where the seams show. Analisa.io built its analytics around Instagram and later added TikTok, and it works on-demand — you search, it computes, you get a picture of that moment. Iconosquare spreads across many platforms at once, TikTok being one tab among several. These are competent products. They’re just answering an Instagram-shaped question — “how big is this account?” — when the TikTok-shaped question is “what’s gaining momentum, and is this creator riding it?” A tool that only shows you a snapshot when you ask is structurally blind to the one thing TikTok rewards.

Problem two: they’re priced to lock out the exact people TikTok rewards

Here’s the part that actually gives it away.

TikTok is the one major platform where a nobody can beat a brand — where a solo founder with a phone can out-reach a company with a media budget. It is, by design, the great equalizer of social.

So why does the software for understanding it cost like enterprise gear?

Look at the published numbers. Exolyt’s entry paid plan is listed at $400 a month, climbing to $950. Pentos has no free plan at all — it starts at $99, and competitor tracking, arguably the whole point, doesn’t unlock until the $299 tier (top plan $999). Iconosquare’s plans start lower, at €33 a month, but then bill €16 per user on top, so your cost climbs every time a teammate needs to look. That per-seat model is a direct import from the Instagram-agency world, where a handful of managers ran a handful of brand accounts.

None of that fits how people actually work on TikTok. The audience TikTok rewards — solo founders, small brands, scrappy agencies — is precisely the audience priced out by a $400 floor and a per-head tax on curiosity. You end up with tools built for the platform where the little guy loses, sold at a price only the big guy can pay, aimed at helping you win on the platform where the little guy is supposed to win. The incentives are backwards from top to bottom.

What “TikTok-native” is supposed to mean

When busypipe says TikTok-first, this is what it’s meant to fix, on both axes.

Technical: it keeps watching the creators, sounds, and hashtags you add and builds history and momentum over time, so you see what’s accelerating — not just a snapshot when you happen to search. Continuous, not on-demand.

Money: it starts free, paid plans run $59 to $299 a month, and there are no per-seat fees — bring the whole team without watching the bill climb per head. Competitor benchmarking is in from the free plan, not gated behind a $299 tier.

I’m obviously not a neutral party here, and you should check every provider’s current pricing and coverage yourself — numbers move, and if you genuinely live across five platforms, a broad multi-platform suite may still be the right call. Fair.

The honest version

The uncomfortable truth about the TikTok analytics market is that a lot of it is Instagram software with a new logo and an old price tag. If TikTok is a side channel for you, that’s fine — the mismatch won’t hurt much. But if TikTok is where you’re actually trying to win, you want a tool built around how TikTok moves and priced for who TikTok rewards. Those are the same test, and most of the category fails both.

Want to see what’s gaining momentum in your niche instead of guessing at a snapshot? Start for free with busypipe.

Most of the social intelligence software out there is priced for Fortune 500 procurement teams. I’m building one that isn’t.

I’ve spent years around social platforms and the same pattern kept showing up: the deepest, most useful tools are locked behind sales calls and five-figure contracts. The teams that need this kind of data the most — small agencies, in-house marketers at growing brands, academic researchers — often can’t justify the spend. So they cobble together screenshots and gut feel. That’s the gap busypipe is built for.

What busypipe is

busypipe is a social intelligence platform. Today it focuses on TikTok. Soon, Instagram and YouTube.

The product is built around one idea: you should be able to track the accounts you don’t own. Competitors. Influencers you work with. Rising creators in your category. Hashtags and sounds that are starting to move in a specific market.

You add the profiles, hashtags, or sounds you care about into a portfolio. The platform then tracks them continuously — follower growth, engagement rate, posting cadence, regional performance, trending audio — and gives you charts and tables you can drop into Monday’s deck.

That’s it. Not a publishing tool. Not a social inbox. Not a 25-platform listening suite. Just deep, organised intelligence on the platforms that actually matter to your audience.

Why I’m building it

Honestly? Because I see the opportunity, I know the field, and I want to earn a living doing something useful.

The big names in this space — Meltwater, Brandwatch, Sprout — are excellent. They are also priced for enterprise buyers. Annual contracts in the tens of thousands. Demos required. Six-month implementations. That works fine when you’re a global PR team. It does not work when you’re a four-person agency running ten client accounts, or a marketing lead at a growing DTC brand, or a PhD student studying how trends move across regions.

I want to make this kind of intelligence accessible. That means flat pricing per workspace instead of per seat. A free plan that’s actually useful. Self-serve sign-up with no demo and no sales call. And a price tag a small agency can put on a credit card without flinching.

There’s a second audience I care about: researchers in the social sciences. Academia needs structured, long-term social data, and right now the options are either enterprise rates they can’t afford or collecting it by hand. I want to serve them too — with a different angle, and with pricing that respects the reality of a research budget.

A note on trust (especially for researchers)

There’s something I want to name up front, because it tends to get glossed over: the numbers a platform shows you today are not the same as what it showed you yesterday — and probably not what it’ll show you tomorrow. They are what the platform chose to expose, on the day it chose to expose it.

Things that worked last month stop working. Numbers move around for no clear reason. Whole categories of data disappear without notice. Anyone who has tried to reproduce a finding six months later knows the feeling.

busypipe handles this by preserving the original snapshot of what the platform showed us, before anything is summarised or scored. If the platform later changes its mind about what it showed yesterday, our copy is still there. The record stays honest even when the source moves.

It’s also why I’d encourage anyone using any social intelligence tool — mine included — to ask: do you preserve the original data, or only the processed numbers? What happens when a platform quietly changes what it shows? If they can’t answer that, treat the numbers carefully.

How to use it

Sign up. Pick a few accounts you care about — competitors, influencers, your own brand. Drop them into a portfolio. Come back in a week.

You’ll see growth curves, engagement trends, post-by-post performance, what hashtags they’re leaning into, what sounds they’re riding, and how their content lands in different regions. Download a spreadsheet when you need to send something to a client.

And soon, you’ll be able to share a live analytics view with anyone  — a teammate, a client, a stakeholder, a thesis advisor — without giving them a seat or asking them to log in. A link is enough. The view stays live and updates as new data comes in.

There’s a free plan for three tracked profiles. Enough to kick the tyres and decide if the product earns a paid slot in your stack.

Where it goes from here

TikTok first, because that’s where the most movement is happening and existing tools are the thinnest. Instagram and YouTube next. Comments, transcripts, shareable analytics, and a layer of post-level intelligence — what kind of hook a video opens with, how strong it actually is — are rolling out as I write this.

If you want to follow along, or kick the tyres, busypipe.com is the place.

If you’re a researcher with a use case I should know about, I’d like to hear from you.

Day five

It’s day five. You’ve cloned the repo. You’ve sketched a half-mental-model of how the auth flow probably works. You’ve started piecing together what the three different folders, all named some variant of “service”, actually do.

Then your Slack lights up.

A teammate just shipped a feature you didn’t even know was in the backlog. A PR opens. Three reviews land in nine minutes. It merges. Another one drops before lunch. By the end of the day, the team has shipped roughly a sprint’s worth of work, and you have written maybe forty lines of code, half of which you’re not sure about.

Welcome to the modern small team. AI in the loop, everyone moving fast, and you, the new hire, feeling like the slowest creature in a room of velociraptors.

And then the question that won’t leave you alone: do I slow down and learn this system properly, or do I lean fully on AI, ship at their pace, and trust the tests, my reviewers, and a bit of luck to keep me out of trouble?

It feels like a binary choice. It isn’t. But to see why, you have to be honest about what each path actually costs.

The turtle path

The first path is the one every senior engineer has historically advised: slow down. Read the code. Trace a request from the edge through the system. Understand the data model before you touch it. Earn the right to write a one-line change by knowing exactly which one line it should be.

This advice is right in spirit and brutal in practice on a fast team. Because while you’re reading, your peers are shipping. Their context grows; yours grows too, but theirs grows on top of new features they just built, while yours grows on a snapshot of the code that is already drifting out of date.

You start to feel a quiet shame around standups. You hedge. You ask questions in DMs instead of channels because you’re afraid they reveal too much. The “I’m just ramping up” line works for a week. After three weeks, it starts to sound like an excuse — to them, but mostly to yourself.

The bigger problem: the codebase you’re trying to learn is changing under you. The README is half a year stale. Half the abstractions are LLM-generated, which means they look like they should fit a pattern but sometimes don’t. By the time you’ve “understood” the system, you’ve understood a system that no longer exists.

This is the part the old advice doesn’t account for: in an AI-native team, the half-life of codebase knowledge has collapsed. Deep, slow learning still works — but the depreciation curve is steeper than it used to be.

The velociraptor path

The other path is to embrace the speed. Use AI the way your peers do. Have it explain the function you’re editing. Have it generate the change. Have it write the tests. Trust CI. Ship it. You’re not pretending to know the system; you’re outsourcing the knowing to the model and the test suite, and you’re betting that the bets your team has already made (good tests, good reviewers, good observability) will catch you when you’re wrong.

For some tasks, this is fine. For some tasks, it’s the only reasonable choice. A small bug fix on a well-tested endpoint, a copy change, a third migration of a pattern that already exists three times, there is no reason for you to spend three days mapping the codebase before doing it.

But here’s the part nobody on the team will say out loud: the model is fluent and confidently wrong, and your reviewers are moving fast too. A pull request can pass with a green check and two thumbs-up and still introduce a subtle issue that only shows up under specific load, or that quietly breaks a contract that wasn’t covered by a test because nobody thought it needed one. AI accelerates everyone, including the people reviewing your code, and acceleration without proportional rigor means errors slip through faster, not less.

There is a version of this path where it works. There’s another version where you wake up one morning to a production incident, look at the diff, and realize: you wrote the code, your reviewer skimmed it, the AI generated the part that broke, and none of the three of you actually understood what it was doing.

Why there’s no universal answer

So which is it? Slow down or speed up?

I don’t think there’s a single answer, and I’ve come to distrust anyone who says there is. It depends on at least five things:

• The blast radius of the system you’re touching. A payments service is not a marketing site. The cost of a wrong AI-generated change on critical-path code is orders of magnitude higher than something you can revert in a click.
• The maturity of the safety net. Strong test coverage, clear ownership, good observability — these mean you can ship fast and learn from production. Without them, “ship fast” is just “break things and hope.”
• Your background. A senior engineer joining a new codebase has stronger pattern-matching priors. They can read a diff and feel when it smells wrong, even without understanding the specifics. A junior doesn’t have that radar yet.
• The team’s actual norms. Some teams say “move fast” and mean it. Others say it and quietly resent the person who breaks production. Watch what actually happens after an incident, not what the README says about culture.
• The task itself. Wiring a new component to an existing pattern is different from designing a new abstraction. The former tolerates speed. The latter punishes it.

The honest move is to treat the slow-vs-fast question as case-by-case , and to be deliberate about which mode you’re in.

A few things that actually help

After watching myself fumble through this, and after talking to other people who’ve gone through it, here is what I’d suggest. Not as rules. As experiments.

Try both modes, on purpose. Pick one task this week to do the turtle way. Read everything around it. Trace it end-to-end. Write the change yourself. Let AI critique it afterwards. Pick another task and do it the velociraptor way — let AI drive, ship as fast as you can responsibly, see what happens in review. Then compare. You’re not choosing a tribe. You’re calibrating your instinct for which mode fits which task.

Use AI as a teacher on the slow path, not just a worker on the fast path. When you’re reading the codebase, asking a model to explain a function, sketch the call graph, or contrast two abstractions is dramatically faster than doing it alone. The turtle path got slower than it needed to be because most of us learned to use AI to compress the writing, not the learning.

Pick a sub-system to actually own. You cannot understand the whole codebase fast. You can pick a corner — a service, a flow, a domain — and become the person who knows it cold within a couple of weeks. That earned ground gives you the confidence to ship fast everywhere else, because you know what real knowledge feels like and you can recognize when you don’t have it.

Watch your taste returning. The signal you’re learning, even on the fast path, is when AI-generated code starts to feel wrong before you can articulate why. Until that feeling kicks in for a given area, you should review every AI suggestion with a slight squint. Once it does, you can move faster there with more trust.

Question the path, not just the line. AI is fluent at the level of code — but it can confidently walk you down the wrong path. You’ll find yourself two hours into an implementation, look up, and realize: this could have been a five-line fix instead of a 200-line abstraction. Or that the model has been “improving” something that didn’t need improving in the first place. The subtle nuances — is this the right problem, at the right level, with the right blast radius? does this even need to exist? — are exactly the parts AI cannot reliably deliver. They require sitting back, ignoring what’s on the screen, and asking whether the direction is honest.

And the inverse is also true. Sometimes the model spots an edge case you never would have considered. It names a failure mode you’d have missed. It points at a library function that does the thing you were about to hand-roll. The posture that works isn’t trust or distrust — it’s a working partnership where the model brings breadth and recall, and you bring the willingness to say no, simpler or no, that’s not the right shape at all. If you give up that role, nobody on the team is playing it.

Pre-commit to a “stop and read” trigger. Decide in advance what kind of change makes you pause: anything touching auth, anything touching money, anything in a file you’ve never opened, anything where the test you’d need doesn’t exist yet. Without a trigger, the speed of your peers will pull you into shipping things you shouldn’t.

Track your own incidents — silently. Keep a private note of every bug you ship in the first three months. What was the cause? Did you understand the code? Did the AI generate the broken part? Did you skip a test you should have written? After ten entries, your pattern is in front of you. It will tell you, more honestly than any retro, where you can safely run fast and where you need to slow down.

What the discomfort actually means

The one I’ve had to keep telling myself: the discomfort of feeling slow is not a signal you’re doing it wrong. It’s the price of caring about the system in a culture that has, sometimes, stopped caring quite as much as it used to. That care is what keeps the team from one day waking up inside a house nobody actually understands.

The right answer probably isn’t turtle. It probably isn’t velociraptor either.

It’s the engineer who knows which one they’re in at any given hour and why.

If you’ve joined an AI-native team recently, I’d love to hear how you handled the first three months. The honest version, not the LinkedIn version.

The most dangerous outages don’t crash.

They go quiet. Throughput drops, but no error page fires. No alarm hits the on-call rotation. The dashboards keep drawing graphs — they just draw flat lines. The system is “up” in every sense the monitoring understands. It’s just not doing any work.

You don’t notice for hours. Then you notice, and you look, and everything claims to be healthy, and there’s a moment when you wonder if the problem is actually you.

This is a story about one of those.

A normal Friday morning

The signal was a steady throughput number — call it ~200 work units per second under normal load — that had collapsed overnight to zero.

Not “near zero,” not “degraded.” Zero.

Then, about two hours later, a burst: 250/s for roughly an hour. Then silence again, six hours of it. Then another burst. Then silence.

The system wasn’t broken. It was breathing. Once every six hours.

If you’ve spent any time in distributed systems, you know the feeling that comes next — the slow, cold realization that the failure mode you’re staring at isn’t a failure at all. It’s the system doing exactly what it was told to do, but at a scale where what it was told doesn’t work anymore.

The shape of the problem

About 50,000 entities feed the work loop. Each one has a “next due” timestamp. The scheduler picks whichever ones are due, processes them, and then reschedules them based on their type and priority. Boring stuff. The kind of thing you write in a weekend and assume you won’t need to revisit.

When I built this three months ago, I picked a parameter that controls how much randomness gets added to those re-scheduling timestamps. The parameter is called spread; its purpose is to keep entities from all coming due at the same instant. I sized it deliberately for the load I expected at launch — generous enough that the re-scheduling looked uniform, tight enough that the budget per cycle stayed reasonable.

What I didn’t size it for was the load three months later, after a 5x growth curve I should have anticipated and didn’t.

At a small scale, the parameter was generous. With ten thousand entities, the spread was wide enough to be effectively uniform.

At fifty thousand, it wasn’t.

What happens when the spread is too narrow for the load? The same thing happens when buses run on a route too short for their headway. They bunch. One bus picks up a few extra passengers, falls slightly behind, picks up more passengers because more have accumulated, and falls further behind. The bus behind it catches up, picks up almost no one, and races ahead. Within a few cycles, you have two buses running together — one full, one empty — and a long, quiet stretch where neither was.

That’s what fifty thousand entities had done to themselves over a few weeks. They’d drifted into a tight cluster. Every time the system processed them, they all became “due” again at roughly the same moment. Twelve hours later: another cluster. Six hours of silence between clusters.

The technical name for this is cohort synchronization. The intuitive name is herd.

The metric that mattered wasn’t on any dashboard

The way I figured this out is worth a paragraph of its own.

The dashboards I’d built showed throughput, error rate, queue depth, and latency — all the things you’d expect. Every one of them was either “healthy” or “consistent with the throughput collapse,” which is to say, not diagnostic. Throughput was zero. Queue depth was zero. Latency was undefined. The error rate was zero. None of those numbers told me why.

The diagnostic metric was a counter — entities removed from the active pool — that I’d never charted, because under steady-state operation it was always zero or near-zero. I queried it because I’d run out of dashboard widgets to look at. It had spiked: tens of thousands of entities removed in a single sweep, then thousands more in the next.

That single line of telemetry was the entire incident.

The lesson is one I’ve now relearned three or four times in my career, and it never sticks until I see it again: dashboards reflect the failures you expected. The metrics that actually diagnose unprecedented incidents are almost always the ones no one charted. You find them by querying everything, not by looking at the wall.

After the incident, the first thing I did was promote that counter to the dashboard. The second thing I did was add four others I’d noticed during the diagnosis but had similarly never bothered to chart. The third thing I did was accept that I will probably do this again next quarter, because incidents teach you about the metrics you should have had, and there is no shortcut to knowing which ones those are.

metrics from busypipe.com

The temptation to rewrite

I want to spend a moment on this, because it’s the part most postmortems leave out.

Spend a day diagnosing a herding bug in your own scheduler, and your hands start to itch. There are mature, off-the-shelf workflow systems. They give you durability, retries, observability, queue management, all the boring infrastructure you keep half-implementing. What if we just used one of those? It’s a real and persistent thought, especially around hour eighteen.

I sketched it out. Honestly. A day-long detour into “what if we migrated.”

The conclusion was the one I should have started with: the herding wasn’t a bug in the scheduler. It was a property of the workload. Migrating to anything off-the-shelf would have been a four-to-six-month project that didn’t address the actual cause. The actual cause was a number — a configuration parameter — that had been quietly too small ever since I’d crossed some threshold of scale, somewhere in the last few weeks. The actual fix was changing that number.

I changed two numbers in two pull requests. Each PR was three new tests and a one-line config change. The tests pin the new behaviour so that the parameter cannot drift back. The PRs were merged the same day they were written.

The lesson: incidents make you want to rewrite. The right answer is almost always to fix the smallest thing that resolves it, pin a test that prevents regression, and write the runbook so the next person doesn’t lose two days. The rewrite belongs on a roadmap, with a budget, after the fire is out — not in the same week as the fire.

Recovery has an order of operations

The textbook story ends when you ship the fix. Real recovery has more steps, and the steps don’t commute.

In my case, three pull requests had to land. One enforced a hard limit on the active pool. One widened the spread for the bulk of the workload. One widened it further for a particular tier of slow-cycling work. Anyone reading the diff after the fact would assume the order didn’t matter much. It did.

If you merged the limit-enforcer first, while the cohort was still bunched, you’d archive thousands of entities from inside the cluster. The ones you kept would all share roughly the same recent timestamp. The herd would reform tighter, not looser.

So the order was: spread first, then run a one-shot SQL operation that broke up the existing cluster, then the limit-enforcer on a now-evenly-distributed pool, then the second spread bump.

I wrote the order into the runbook before I ran the steps, so future-me — possibly sleep-deprived, possibly mid-incident — would not have to derive it under pressure.

Real recovery is a sequence with a footnote at every step. The fix isn’t the diff; the fix is the diff plus the order plus the reason.

The “this looks broken but isn’t” footnote

While writing the runbook, I added a warning that I’m proud of. It said, in effect: during the drain, the system will look broken. Some entities will appear overdue. Throughput will lag. Don’t roll back. Don’t panic. This is the recovery; it looks like the failure for the first four to eight hours.

The reason that the warning is in the runbook is that I almost rolled the fix back, twice, while the recovery was running. The metrics were temporarily worse in shape than they had been pre-fix — exactly the way you’d expect a queue under structural rebalancing to look — and every reflex I have was screaming to revert.

The runbook is written by people who imagine themselves anxious. A runbook that only tells you what to do but not what it’ll look like is a runbook that gets ignored at exactly the moment it was meant to be useful.

I now write a “what to expect, including the scary parts” section in every runbook I touch.

What I’d want a peer to take from this

If you’ve read this far, you’ve probably had a Friday like mine. A few things I want to keep, written down so I have a chance of remembering them:

The most expensive incidents are the quietest ones. The system that crashes pages everyone instantly. The system that goes silent gives you two days of a sinking feeling and then a long diagnosis. Build alerts on zeroes, not just on errors. A throughput of zero is an event.

The metric that diagnoses a novel incident is the one that no one charted. Query the long tail of your telemetry first. Promote whatever you find afterwards.

Resist the rewrite. Fix the parameter. Pin the test. Then put the rewrite on a roadmap with a budget and a quarter, not on a Monday with a coffee.

The order of operations during recovery is part of the fix. Write it down before you run it.

Real runbooks describe what the recovery looks like, not just what it does. Including the parts that look like the original problem.

Time-to-understand is usually much longer than time-to-fix. Once I understood, the code change was about ninety minutes of work. Understanding took two full days. That ratio is normal, and it’s easy to plan for the ninety minutes and forget the two days. That’s why the same incident keeps recurring in the same codebase under the same engineer.

Two quiet days of production cost very little in user-visible terms. The system continued to “run.” The graphs continued to draw. Some work happened in bursts.

The cost of not understanding what I’d just lived through would have been a third quiet outage a few weeks later — when neither I nor the runbook would be ready, and I’d start the diagnosis from zero again.

The system has recovered. The numbers are bumped. The tests are pinned. The runbook is on disk.

Next time it goes quiet, I’ll know how to listen.

The four-week markdown file

There’s a file on my laptop that doesn’t exist yet.

It’s called loki-health-report.md. It has three sections: green, degraded and broken. The whole job is to open the file, type three headings, write one bullet under each, and commit it. Fifteen minutes of work, tops.

I have not done it for four weeks.

I know it’s been four weeks because my planning system told me. Last night’s daily plan had the block written out in full:

- [] 20:00 – 20:30 Loki report — open the file 📄 (4× CARRY — last chance this week)

Next to it, in my own handwriting from that morning, was a rule: “If skipped again tonight, mark as ‘drop this week’ in tomorrow’s daily — stop carrying a 4× block.”

I skipped it. Again.

Then I opened today’s plan and wrote the words I had promised myself I’d write: Drop it from the week.

This is, I think, the only reason the system works at all.

What most productivity systems hide

I’ve tried most of them. GTD. Bullet journaling. Todoist. Things. Notion databases with rollups and Kanban views and “this week” smart filters. I kept a to-do list for about a decade.

They all have the same structural failure mode: they hide how long a task has been sitting there.

When a task moves from Monday to Tuesday in a to-do app, nothing visible happens. The checkbox is still unchecked. The task is still “open.” Your list still looks like a reasonable amount of work. You feel, at worst, mildly behind.

But mild, extended, invisible avoidance is how real opportunities die.

A task that sits undone for four weeks isn’t a task anymore. It’s an aversion, and the aversion has a shape — it tells you something about what you’re afraid of, or what’s genuinely unimportant, or what you secretly disagree with yourself about. A to-do list with a four-week-old item just looks like a to-do list with one more item on it.

Most systems are built to make you feel organized. I wanted one that would make me feel honest.

The core mechanic: make the carry visible

The system I use now is time-blocking with one small, uncomfortable modification: a carry counter.

Every task has a target day. When a task doesn’t get done, it doesn’t just get “rolled forward.” It goes into the next week’s plan with a number next to it: carried 1×, 2×, 3×. In the weekly review, I write the carry count in the task line itself:

- [] 🔥 Loki API health report — open a .md file and write 3 sections (carried 3+ weeks)

- [] salpon.com — terms + privacy pages (carried since week-14; 4 weeks stale)

- [] IBKR/Nordest business account (carried since week-13; 5 weeks)

That’s a real snippet from last week. You cannot look at that list and feel organized. You look at that list and feel something closer to shame — which, it turns out, is more useful.

The counter has one job: make it impossible to pretend a task is “on my list” when the truth is it’s been on my list for five weeks, and I have no plan to do it. The moment it becomes countable, two things happen:

1. You actually do the easy ones (the act of writing “4×” next to open a file and type three headings is a very effective inner alarm).
2. You finally kill the ones that never belonged there.

Most of my carried tasks don’t survive three carries. They either get done or they get dropped — with a note about why they’re being dropped, which becomes its own lesson.

The shadow side is real. Carry counts sting. A high carry count means I’m avoiding something. But that’s the feature. Avoidance has to be seen before it can be decided about.

The skeleton underneath

The carry counter only works because it sits on top of a rigid planning skeleton. Without the skeleton, “I didn’t do it” is ambiguous; did I not get to it, or was it not really planned? The skeleton removes that ambiguity.

Four levels, each feeding the one below it:

Yearly. Vision and the two or three things that matter this year. Written once, glanced at monthly.

Monthly (e.g. 2026-04.md)  . Two or three focus areas, the projects they map to, and the habits I want to defend. When April started, the focus was ship BusyPipe to live monitoring, keep TopTop steady, don’t break the Swedish streak.

Weekly (e.g. week-17.md)  . The one that does the actual work. Each weekly file has two halves: a prologue with three priorities, hard commitments, carried items with their counts, and the rules that govern the week — and an epilogue written seven days later, reviewing what happened. The prologue is where the carry counter lives.

Daily (e.g. 2026-04-22.md)  . A pre-committed schedule in 15–90 minute blocks, in plain markdown:

- [] 08:00 – 09:00 Communication Course — final pass + send-prep
- [] 09:00 – 09:30 class setup + breakfast
- [] 09:30 – 14:00 Swedish intensive (calendar, hard)
- [] 14:00 – 14:45 lunch + walk
- [] 15:00 – 16:30 Communication course — SEND
- [] 17:45 – 20:00 family evening (protected)
- [] 20:00 – 20:30 Loki report — open the file (4× CARRY)

At day’s end, every block gets marked: ✅ done or 🔴 failed. No “partial,” no “rescheduled,” no “kind of.” Binary.

This sounds harsh. It is harsh. But “partial” is what lets tasks live on a list for four weeks.

The rules that emerged

I didn’t design this system in one sitting. I started with time-blocking and ✅/🔴 tracking in August 2025. The rules came from patterns in the epilogues, week after week of the same tasks slipping for the same reasons.

These are the rules that survived eight months of weeks:

• Max three focused tasks per day. Not three things — I still have a dozen blocks on a typical day, including lunch and Swedish and family evenings. But only three of them are focused work that demands cognitive load. Any more and the fourth reliably becomes a 🔴.
• Morning blocks are sacred. 09:00–12:00 is deep work. No meetings, no admin, no “quick emails.” Violating this rule costs the rest of the day.
• Evening blocks are recovery — not deliverables. This rule took me six months to accept. I used to plan evening “bonus” work blocks that hit maybe 15% completion. Now evenings are for admin, reading, and family, and any work that happens is genuine surplus rather than scheduled guilt.
• Written report blocks start with touch. If a block says “ship a report,” the first physical action is to create the markdown file and type the section headings. I learned this from the Loki report. For three weeks, the block kept turning into “build more tooling instead of writing.” The rule is now literal: step one is touch name.md.
• One-shot attempts for time-sensitive blocks. Some blocks depend on a specific window (a morning call, a testing hour, a shipping deadline). If the window fails, the task drops from that week, it does not reschedule. This forces honest planning for next time.
• Weekends are zero scheduled knowledge work. Family, physical projects, recovery. When I break this rule, the following Monday reliably breaks.

Every rule here started as a 🔴 that kept happening. The system doesn’t give you the rules. It just makes it humiliating enough to ignore them that you eventually write your own.

What actually changes

The measurable change isn’t that I get more done. Some weeks I get more done, some less. The change is in what I’m doing.

I kill things faster. The marketing strategy document for my primary product sat on my plan for a month. Every week, I scheduled a block for it. Every week, the block got spent doing something else instead, usually building more tooling. In week-16, after two protected blocks and roughly nine hours burned, I wrote a rule: “permanent drop. The deliverable shape was wrong.” That was a real insight, and I only got to it because the system forced me to watch myself fail the same way five weeks in a row.

I notice when the plan is lying to me. On Wednesday, April 22, the week-17 plan had scheduled a gym slot at 11:00 and a deep work block. But my calendar had a Swedish class from 09:30–14:00 that I’d forgotten about. That contradiction made it into the morning notes — “this breaks the planned Wed shape; salpon.com AM block is GONE, gym 11:00 is GONE” — and I rescheduled instead of pretending. Without a written plan, there’s no contradiction to notice.

I can see my avoidance shape. My carried tasks clump. They are almost always: admin (opening business accounts), writing (Medium posts, reports), and publishing (LinkedIn posts, marketing comms). That is diagnostic information. I don’t procrastinate on building things. I procrastinate on being visible and on paperwork. That tells me more about myself than a decade of to-do lists did.

I stop pretending I’m going to get to it. The clearest experience of the system is the one that started this post: knowing, in advance, that I’m about to drop a task. The 4× Loki report line told me yesterday morning that tonight was the deadline and the drop was probable. When I skipped it, there was no surprise. There was only the calm, honest act of writing “drop this week” in today’s plan.

That’s not a productivity win. It’s a dignity win. It replaces the low-grade guilt of an open list with the clean decision of closure.

How to start tomorrow

If you want to try it, don’t adopt the whole thing. Adopt three things:

1. Pick a file format, any file format — but one file per week and one per day. Mine is Obsidian markdown, because the files are plain text and will still be readable in 20 years. Notion, a physical notebook, Apple Notes — anything that forces weekly and daily artifacts. Not an app with a smart-filter view. You need files, not queries.
2. Time-block the day, the night before or the morning of. Every block gets a start and end time. Every block gets marked ✅ or 🔴 when it ends. No third option.
3. At the end of the week, carry unfinished tasks into next week’s plan with a carry count. Increment it every week. When a task reaches 3×, it must either get done or get dropped with a written reason. No task is allowed to live at 4× or above.

That’s it. That’s the whole system. It doesn’t require an app, a subscription, or a methodology. It requires a text file, two symbols, and the willingness to let a number next to a task tell you the truth about what you’re doing.

The file that doesn’t exist yet is still on my laptop. It’s called loki-health-report.md. Today it got dropped. Next week it will not be on my list — because this morning, before anyone else was awake, I wrote the word “drop” in a markdown file, and the number next to it stopped going up.

That is what a productivity system is for.

If you run something similar, or something better, I’d love to hear how you handle the carry. Writing about it is, in itself, one of the items that kept getting carried on my list. This post is a 3× that finally shipped.

I’ve been using Obsidian daily since August 2025. Daily plans, project specs, business ideas, speech scripts, study notes, expense tracking — 273 markdown files across a dozen folders.

And I couldn’t find anything.

Not because Obsidian is bad at search. It’s excellent. The problem was me. My notes were written for the version of me who was sitting at the keyboard that day. Future me had no idea what “check the thing for bp” meant three months later. My ideas/ folder was a graveyard of half-sentences. My study/ folder had two checklist items pretending to be course notes.

Sound familiar?

The Karpathy Approach

Then I read Andrej Karpathy’s post about using LLMs to build personal knowledge bases. His approach is elegant:

1. Ingest raw sources (articles, papers, repos, notes) into a raw/directory
2. Compile them with an LLM into a wiki — a collection of interconnected .md files
3. Query the wiki by asking the LLM complex questions against it
4. File back the answers and explorations into the wiki, so it always grows

The critical shift: you rarely touch the wiki directly. It’s the LLM’s domain. Your job is to feed it sources and ask questions. The LLM organizes, cross-links, and maintains everything.

I realized I was sitting on 8 months of raw source material. It was already in markdown. It just needed compilation.

What I Actually Did

I sat down with Claude in a single session and went through this process:

1. Audit

First, I had Claude read through every folder in my vault — projects, ideas, study notes, business files, daily plans, templates. Not skimming — actually reading the content and cataloguing what knowledge existed, how detailed it was, and what could be extracted as standalone wiki articles.

The results were humbling. My study/ section was three files containing a grand total of five lines. My ideas/ folder had seven files averaging about 120 bytes each. But my daily plans? 220+ files of meticulous time-blocked schedules with real insights buried in the notes sections. My Toastmasters speech scripts had genuine narrative wisdom in them. The knowledge was there — it was just trapped in the wrong format.

2. Design the Structure

We designed a wiki structure with six domains:

• Tech  — projects, tools, infrastructure
• Business  — ventures, finances, consulting
• Career  — university, speaking, professional growth
• Learning  — courses, languages, research
• Personal  — productivity, health, routines
• Ideas  — explorations and “what if” thinking

Plus a concepts/ folder for cross-cutting themes that connect multiple domains — things like “Time-Blocking” (which touches personal, business, and tech) or “Multiple Income Streams” (connecting business, career, and personal philosophy).

3. Compile

This is where the magic happened. Claude read my raw notes and compiled them into 44 structured wiki articles, each following a consistent template:

• A one-line summary
• Status indicator (Active, Dormant, Stub)
• Source attribution back to the original note
• Related article links
• The actual content — not copied, but *distilled*
• Open questions for future exploration
• Cross-links to related articles

For example, my scattered BusyPipe project notes (spread across 8 files including Excalidraw diagrams and draft schemas) became a single clean article that explains what BusyPipe is, its architecture, milestones, and how it connects to my other projects and financial strategy.

My Toastmasters speech about failure and financial independence got compiled into a concept article called “Learning vs Racing” that connects to my business strategy, my content creation plans, and my productivity philosophy.

4. Build the Pipeline

A wiki is only useful if it keeps growing. So we built three maintenance mechanisms:

Raw ingest folder (wiki/raw/) — A place to drop web clips, papers, screenshots, meeting notes. Anything. The LLM reads them and compiles them into proper wiki articles on request.

A /wiki-write skill  — At the end of any conversation with Claude, I type “save to wiki” and it extracts the key insights, conclusions, and decisions from the entire conversation and saves them as a structured markdown file in raw/. No more losing knowledge that only existed in a chat window.

A weekly health check  — An automated task that runs every Sunday night and scans the entire wiki for broken links, stale articles, gaps between what I’m working on (from daily plans) and what the wiki covers, and missing cross-links. It writes a report and fixes simple issues automatically.

What Changed

The immediate difference: I can now ask Claude “what’s the relationship between my BusyPipe project and my SaaS costs?” and it navigates the wiki, finds the relevant articles, and gives me a coherent answer that connects my expense tracking, my subscriptions, my TopTop revenue, and my financial independence strategy. It can do this because the knowledge is structured and cross-linked, not scattered across random files.

But the deeper change is philosophical. My notes used to be write-only — useful the day I wrote them, then slowly decaying into noise. Now every note, every conversation, every web clip feeds into a living knowledge base that gets richer over time. Karpathy’s phrase captures it perfectly: explorations and queries always “add up.”

wiki graph

How You Can Do This

You don’t need anything fancy. Here’s the minimum:

1. An Obsidian vault (or any folder of markdown files) — you probably already have this
2. An LLM with file access  — Claude with Cowork mode, Cursor, or even the API with a script
3. One session to do the initial compilation  — have the LLM audit your notes, design a structure, and compile articles
4. A raw/ folder for ongoing ingestion
5. The Obsidian Web Clipper extension for saving web articles as markdown

The key principle: stop thinking of your notes as finished products. They’re raw material. Let the LLM compile them into something structured, and then keep feeding it more raw material.

Your scattered notes are a goldmine. You just need a smelter.

I built this using Claude in Cowork mode with an Obsidian vault mounted as a workspace folder. The entire process — from audit to 44 compiled wiki articles — took a single afternoon session.

In fact, I had been thinking about this for a long time, but like any engineer, I didn’t have the time. https://www.linkedin.com/posts/activity-7400255931525271552-eknz

The Gap Nobody Automates

There are thousands of productivity tools. Calendar apps, task managers, habit trackers, Pomodoro timers, note-taking apps, and AI assistants. I’ve tried most of them.

They all break at the same point: the handoff.

You plan in one place. You work in another. And between the two, there’s a gap where good intentions go to die. You wrote a beautiful time-blocked schedule at 8 am. By 10:15, you’ve forgotten it exists.

I use Obsidian for everything — project notes, weekly goals, meeting minutes, and daily plans. It’s the best thinking tool I’ve found. But thinking and doing are different activities. Obsidian is exceptional at the first and silent about the second.

What I wanted was a system that could:

1. Read my notes, goals, and yesterday’s leftovers
2. Generate a realistic daily plan — time-blocked, context-aware
3. Watch that plan file all day and remind me when it’s time to switch tasks

No manual prompting. No copy-pasting. No switching between apps to check what’s next.

So I built it.

The Three-Tool Stack

The system has three components, connected through plain markdown files:

Claude (planning agent)
↓ writes
Obsidian Vault (source of truth)
↓ watched by
DayWatch (notification engine)

Claude: The Planner

Every morning at 8 am, a Claude Cowork scheduled task runs against my Obsidian vault. It reads:

• My weekly plan (goals, priorities, commitments)

• Yesterday’s daily note (what got done, what didn’t, any notes I left for tomorrow)

• My calendar (via Google Calendar MCP)

• Any flagged items from project notes

From this context, Claude generates a time-blocked daily plan in markdown:

markdown
## Day Planner
- [] 08:30–09:00 planning & review
- [] 09:00–11:00 deep work: API refactor
- [] finish auth middleware
- [] write integration tests
- [] 11:00–11:45 gym
- [] 12:00–12:30 lunch
- [] 12:30–13:00 email & slack triage
- [] 13:00–15:30 project work: DayWatch release
- [] 15:30–16:00 Swedish practice
- [] 16:00–17:00 reading & review

This file lands in my vault at plans/2026/03/2026–03–26.md. It’s just a markdown file. I can edit it in Obsidian, tweak the times, add tasks, and check things off as I go.

The key insight: Claude isn’t generating a generic schedule. It’s reading my actual context  — what I committed to this week, what I failed to finish yesterday, what meetings are on my calendar — and producing a plan that accounts for all of it. The quality of AI-generated plans scales directly with the context you give them.

Obsidian: The Source of Truth

Everything lives in the vault. Plans, notes, goals, templates — all plain markdown, all version-controlled, all searchable.

Obsidian is the hub that both Claude and DayWatch connect to. Claude reads from it and writes to it. DayWatch watches it. I edit it. Nobody needs an API. Nobody needs a sync service. The filesystem is the API.

This is deliberate. I’ve been burned by productivity tools that lock your data behind proprietary formats or cloud services. When the company pivots, raises prices, or shuts down, you lose everything. Markdown survives.

DayWatch: The Accountability Layer

DayWatch is the piece I had to build because it didn’t exist.

It’s a system tray app (800 lines of Python) that watches my daily plan file and sends native desktop notifications when time blocks are about to start. Click the tray icon and you see your day at a glance:

Example of a daily plan. (PS: My Saturdays don’t look like this 😅.)

Five minutes before each block starts: a notification. When the block starts: another notification. If I launch the app mid-block: an immediate “you should be doing X right now” notification. If I edit the plan in Obsidian: instant reload, rescheduled notifications.

It’s built with Claude Code, which is ironic and appropriate — Claude plans my day, and Claude built the tool that enforces the plan.

Why This System Doesn’t Exist Yet

I searched extensively before building DayWatch. Here’s what’s out there:

Obsidian Day Planner plugin  — adds a visual timeline to your daily notes. Great for seeing your schedule. Doesn’t send notifications. The plan stays inside Obsidian; if you’re not looking at Obsidian, you’re not looking at your plan.

Obsidian Reminder plugin  — sends notifications for tasks annotated with @date reminders. Closer to what I needed, but requires you to manually tag each task with a reminder time. It doesn’t understand time block ranges (08:00–09:00) — only individual timestamps. And critically, it only works while Obsidian is open. Close Obsidian, and your reminders stop. There’s no standalone tray presence, no progress tracking, and no “coming up in 5 minutes” lead notifications.

NotePlan (~$108/year) — the closest commercial alternative. Markdown-based, supports time blocks, sends notifications, and has a menu bar presence. But it’s a full proprietary editor, not a file watcher. It doesn’t watch external markdown files from your Obsidian vault — it expects you to work inside NotePlan. That breaks the workflow where Claude writes to your vault and you edit in Obsidian.

Claude/ChatGPT for planning  — people ask AI to generate schedules all the time. But it’s a one-shot interaction. You prompt, you get a response, you copy-paste it somewhere. There’s no loop. No context. No follow-through mechanism.

Calendar apps with notifications  — Google Calendar will remind you about meetings. But it doesn’t know about your Obsidian notes, your weekly goals, or the task you didn’t finish yesterday. And entering time blocks into a calendar is friction that kills the habit.

Scheduled AI tasks  — Claude Cowork’s scheduled tasks are new and powerful. People are using them for morning briefs, email summaries, and report generation. But I haven’t seen anyone connect the output to a notification system that watches the generated file.

The gap is always in the same place: between the planning tool and the execution tool. Nothing watches an external markdown file for time blocks and sends native OS notifications as a standalone tray app. That’s the piece I had to build.

What I Learned

1. Context is everything for AI planners

When I first asked Claude to “make me a daily plan,” it produced something generic and useless. When I pointed it at my weekly goals, yesterday’s daily note, and my calendar, it produced something I’d actually follow. The difference isn’t the model — it’s the context.

If you’re going to use AI for planning, give it your real notes. Not a prompt. Your actual, messy, evolving notes.

2. Files are the best integration layer

Every component in this system talks through the filesystem. Claude writes a markdown file. DayWatch watches it. Obsidian edits it. No APIs, no webhooks, no sync services. Just files.

This makes the system trivially debuggable (open the file, read it), trivially extensible (any tool that reads markdown can join), and trivially portable (copy the folder, everything works).

3. The notification is the product

I spent years planning in Obsidian. The plans were good. My follow-through was terrible. Adding a single notification — “Deep work starts in 5 minutes” — changed my completion rate more than any planning system, template, or methodology ever did.

The plan was never the bottleneck. The reminder was.

4. Automation compounds

Day one: Claude generates a plan, I follow it better because of notifications.

Day seven: Claude notices I consistently skip the 16:00 block and moves it earlier.

Day thirty: Claude knows my patterns, my projects, my energy levels. The plans get better because the context gets richer.

A human planner would take weeks to learn your rhythms. An AI planner reading your vault has them on day one.

How to Set This Up

The DayWatch part (open source)

# Install
daywatch init - vault ~/your-obsidian-vault
daywatch config # set vault path and plan pattern
# Run
daywatch run

GitHub: [github.com/DiyazY/daywatch]

The Claude planning agent part

In Claude Cowork, create a scheduled task that runs every morning. Point it at your vault. Give it a prompt like:

> Read my weekly plan at weekly_plans/current.md, yesterday’s daily note, and my calendar. Generate today’s time-blocked daily plan at plans/2026/03/2026–03–26.md. Account for unfinished tasks, energy patterns, and today’s meetings. Use the Day Planner markdown format with - [] HH:MM — HH:MM label syntax.

That’s it. Claude handles the rest. DayWatch picks up the file and starts notifying.

The Bigger Picture

We’re in an odd moment for productivity tools. AI can generate plans, summarize notes, and organize information. But the output usually ends up in a chat window that you close and forget.

The missing piece isn’t smarter AI. It’s connecting AI output to the real world — to your filesystem, your desktop notifications, your actual workflow. The tools that win won’t be the ones with the best models. They’ll be the ones that close the loop between thinking and doing.

For me, that loop is: Claude thinks. Obsidian stores. DayWatch reminds.

It’s three simple tools, connected by plain text, doing something that no single app does alone.

DayWatch is open source under MIT. It works with any markdown editor, not just Obsidian. If you plan your days in plain text, it’s the missing notification layer.

GitHub: [github.com/DiyazY/daywatch]

I’m not a gambler. I’m a builder. And when a question like that gets stuck in my head, I don’t Google the answer — I build the answer.

So I did.

It Started With a Simple Question

Eurojackpot has been running since 2012. That’s over 900 draws. Thousands of numbers pulled from that machine, twice a week, for over a decade. All of that data is just… sitting there. Public. Free.

I kept thinking: what if I could see which numbers come up more often? Which ones are “overdue”? Are there patterns in the day of the week? What about number pairs — do certain combinations show up together more than others?

I wasn’t trying to crack some code. I know the math. Every draw is independent. The ball doesn’t remember where it landed last Tuesday.

But here’s the thing — even if you can’t predict the future, you can make more informed choices. And honestly? It’s just way more fun to play when you have data backing your picks instead of your birthday.

So far, I am negative 18e 😅

From Curiosity to a Full Product

What started as a weekend experiment turned into something I actually use every week. And then I thought — if I find this useful, maybe other players would too.

So I built it into a proper web app: eurojackpotstats.eu

Here’s what it does, in plain English:

• It analyzes every Eurojackpot draw ever played. Hot numbers, cold numbers, overdue numbers, trends going up, trends going down — it’s all there, visualized and easy to understand.
• It generates number combinations using 6 different strategies. You can go pure random, lean into the most frequent numbers, pick the coldest ones (contrarian style), or use a balanced mix. Each strategy has a logic behind it, and you can see exactly why it picked what it picked.
• It tracks your picks against real results. Save your combinations, assign them to upcoming draws, and after the draw happens — boom, you see your matches instantly. Greens for hits, reds for misses. No manual checking.
• It even tracks your spending. Set your ticket price once, log any wins, and the dashboard shows you the honest picture. Total spent. Total won. Net position. That line usually goes in one direction, but at least you can see it clearly.
• And my favorite recent addition — a Strategy Scorecard. I ran thousands of Monte Carlo simulations to test how each of the 6 strategies would have performed against the last 20 real draws. It’s a leaderboard of strategies, ranked by actual performance. Does the “hot numbers” approach actually beat random? Now you can see for yourself.

The Details I’m Proud Of

The tool updates itself. New draw results get pulled in automatically — I don’t have to lift a finger. It’s been running on its own for weeks now, and every time I open it, the latest draw is already there.

It supports 22 languages. Every country that plays Eurojackpot can use it in their own language. Dates, day names, everything is localized. A player in Finland sees Finnish. A player in Spain sees Spanish. I’m weirdly proud of this.

There’s also a full advanced analysis page for the data nerds (like me). Number pair frequency. Position analysis — which numbers tend to appear in each sorted slot. Sum range distributions. Trend analysis comparing recent performance against all-time averages. It goes deep.

Why I’m Sharing This

I built this for myself. I play Eurojackpot most weeks, and I wanted a tool that respected my curiosity without insulting my intelligence. No “guaranteed winning systems.” No paid predictions. Just data, clearly presented, with honest tools to explore it.

Is it going to make me rich? Almost certainly not. The odds of hitting the jackpot are 1 in 139 million. I know that. You know that.

But picking numbers backed by actual statistical analysis, tracking my results over time, and seeing exactly where my money goes — that makes the game more interesting. It turns a blind guess into an informed one. And it turns a passive habit into something I actively engage with.

If you play Eurojackpot, give it a look: eurojackpotstats.eu

It’s free. No sign-up required to explore the stats. Create an account if you want to save combinations and track results.

And if the data helps you win something — I’d love to hear about it.

I’m a solo builder shipping tools I actually use. If you’re into building in public, data-driven side projects, or just lottery stats, let’s connect.

Please gamble responsibly. This is a statistical analysis tool for entertainment purposes — not a guarantee of winning.

Speculative Design | Fart Reactor

1.From Taboo to Technology : Why farts might be worth harnessing.

2.Inside the Fart Reactor : A look at the hypothetical mechanics.

3.Breaking the Silence : Social and cultural impacts of body-powered devices.

4.Beyond Farts : Other human-based bioenergy innovations.

5.Design Challenges : Comfort, efficiency, and privacy concerns. 👈

6.Ethics and Ownership : Who controls the data tied to our bodily by-products?

7.Speculative Futures : Where human-powered tech could lead us next.

Comfort And Wearability

https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdatawrapper.dwcdn.net%2Fg5NP2%2F1%2F&display_name=Datawrapper&url=http%3A%2F%2Fdatawrapper.dwcdn.net%2Fg5NP2%2F1%2F&image=https%3A%2F%2Fdatawrapper.dwcdn.net%2Fg5NP2%2Fplain-s.png%3Fv%3D1&type=text%2Fhtml&schema=dwcdn

Energy Efficiency

Even a raincoat’s worth of sweat yields only milliwatts. In practice, the tiny currents from body-powered gadgets fall far short of a phone or smartwatch’s needs. For instance, researchers developed a printable sweat-powered biofuel patch that ran a small lactate sensor and Bluetooth link. In tests it produced about 4.3 mW at 3.66 V 4 — enough to power a fitness meter for 1½ hours from just a drop of sweat 4. That sounds promising, but a basic activity tracker still needs 10–20× more power (typically ~1–2 mW) to run continuously [5]. Likewise, body-heat generators (thermoelectric modules) can scavenge some watts, but only with bulky heat sinks and only when skin is uncovered . In short, human-harvested power is real but meager [5].

Boosting efficiency usually means tradeoffs. To squeeze more juice out of sweat, scientists have “3D-engineered” electrodes — growing carbon-nanotube sponges to expose far more enzyme-catalyst surface to the body’s fluids [5]. Other teams weave micro-TEGs into clothing or add tiny motion harvesters in shoes. Every tweak adds complexity (and sometimes stiffness), so researchers are also working on smarter power management: buffering energy in thin-film supercapacitors, duty-cycling sensors, and offloading data bursts only when needed. In the lab this has raised performance — one headband fuel cell reached ~100 μW/cm² [5] (about twice what a small heat-driven TEG could muster under clothing). But real-world wearables still run out of power within hours, not days.

Data and Privacy

Perhaps the biggest hurdle is that these devices must know everything about you to work — and that scares people. Modern wearables gather heart rate, motion, sweat chemistry, even core temperature. Employers are already using armbands and patches (similar to lab prototypes) to flag heat-stress in workers [6]. The problem is data retention. In one report, companies kept years of biometric logs on employees — and privacy advocates warn it can be abused. As one expert put it, long-term health data could let bosses “kick an employee off a health plan or fire them” [6]. In other words, a life-tracking gadget might double as a surveillance tool.

To address this, experts urge strict safeguards. Workers should be allowed to opt in (or out) of monitoring, and the sensor platform must process only the data it truly needs and delete it quickly [6]. Some designers propose on-device encryption or edge-computing so raw metrics never leave the device. In speculative designs, people imagine fuel-harvesting “smart clothing” that never broadcasts sensitive data — instead computing alerts locally (e.g. a vibration warning for overheating) [6]. Privacy-by-design will be crucial: without it, even the coolest self-powered gadget will feel too intrusive to wear every day.

Harnessing human power is a poetic idea: every step, every heartbeat, every breath could keep our devices alive. But until designers solve the “big three” — comfort, efficiency, and privacy — these gadgets risk being novelties rather than necessities. The future will likely come not from squeezing out one more microwatt, but from rethinking the contract between humans and machines: how much energy we can give, how much data we should share, and how much discomfort we’re willing to endure. Speculative design reminds us that technology is never just technical — it’s also ethical, social, and deeply human.

Speculative Design | Design Challenges: Comfort, efficiency, and privacy concerns

References

[5] Why Sweat Will Power Your Next Wearable

[6] Sensors can read your sweat and predict overheating. Here’s why privacy advocates care

Speculative Design | Fart Reactor

1.From Taboo to Technology : Why farts might be worth harnessing.

2.Inside the Fart Reactor : A look at the hypothetical mechanics.

3.Breaking the Silence : Social and cultural impacts of body-powered devices.

4.Beyond Farts : Other human-based bioenergy innovations. 👈

5.Design Challenges : Comfort, efficiency, and privacy concerns.

6.Ethics and Ownership : Who controls the data tied to our bodily by-products?

7.Speculative Futures : Where human-powered tech could lead us next.

1. Urine-Based Electricity: Microbial Fuel Cells (MFCs)

The Technology
Microbial fuel cells use microorganisms to break down organic matter — in this case, urine — and produce electricity in the process. The bacteria metabolize compounds found in urine, releasing electrons that generate a measurable electrical current.

Evidence & Real-World Application

• Researchers at the University of the West of England have successfully developed “Pee Power” urinals capable of generating enough electricity to power LED lights and small mobile devices, demonstrating genuine practical application of MFC technology (Ieropoulos et al., 2013; Walter et al., 2016).
• The success of Pee Power technology at Glastonbury Festival(a large music event in the UK) showed how microbial fuel cells could effectively function at scale, capturing electricity from festival-goers’ urine to power lighting (Ieropoulos et al., 2016).

Impact
MFCs could play a pivotal role in rural or disaster-hit regions, offering affordable sanitation solutions while generating off-grid electricity.

2. Sweat Power: Biofuel Cells in Wearables

The Technology
Sweat contains lactate, glucose, and electrolytes — substances biofuel cells can convert directly into electrical energy. Wearable sensors and small medical devices can thus draw power directly from human sweat.

Evidence & Real-World Application

• Researchers at the University of California San Diego created a wearable patch capable of generating continuous electrical power directly from sweat to power health sensors (Bandodkar et al., 2017).
• Sweat-powered biofuel cells have demonstrated sufficient power to continuously drive biosensors and Bluetooth communication modules in wearable devices, making battery-free, self-sustaining wearable tech feasible (Yu et al., 2020).

Impact
Sweat-powered devices can significantly extend the life and reduce the environmental waste associated with disposable batteries, making wearable technology truly sustainable.

3. Body Heat: Thermoelectric Generators

The Technology
Thermoelectric generators (TEGs) convert heat differences into electrical energy. When applied to wearable tech, the difference between body heat and ambient temperature generates small but consistent amounts of electricity.

Evidence & Real-World Application

• Scientists at North Carolina State University developed flexible thermoelectric devices capable of harvesting body heat to power wearable electronics, such as fitness trackers or health monitoring patches (Kim et al., 2014).
• Seiko introduced a wristwatch powered solely by body heat, highlighting the commercial viability and practical integration of thermoelectric generation into consumer products (Leonov, 2013).

Impact
TEGs harness otherwise wasted thermal energy, offering continuous, battery-free solutions for personal electronics and medical wearables, enhancing sustainability and user convenience.

4. Motion and Footsteps: Piezoelectricity

The Technology
Piezoelectric devices convert mechanical energy, such as footsteps or muscle movements, into electrical energy through material deformation.

Evidence & Real-World Application

• Pavegen tiles installed in cities worldwide capture energy from pedestrian footsteps, powering streetlights, sensors, and signage, demonstrating viable urban renewable energy (Pavegen, 2023).
• Harvard researchers developed piezoelectric clothing capable of generating electricity from everyday bodily motions like walking or breathing (Dagdeviren et al., 2014).

Impact
Piezoelectric energy capture integrates seamlessly into daily routines, providing passive, continuous energy to support urban infrastructure and wearable technology.

Why These Innovations Matter

Human-based bioenergy innovations, while initially strange or speculative, underline a crucial shift: what we consider waste or mere bodily functions can become valuable renewable resources. Such innovations:

• Enhance sustainability : Reducing reliance on disposable batteries and fossil fuels.
• Improve public health : Enabling battery-free medical sensors and accessible sanitation.
• Transform cultural perceptions : Encouraging more comfortable, pragmatic discussions around bodily functions and sustainability.

These technologies demonstrate that speculative concepts like our “fart reactor” aren’t as far-fetched as they first appear. Instead, they sit alongside genuine scientific and commercial efforts, pushing us toward a more integrated, sustainable future.

References And Further Reading

• Ieropoulos, I. A., Greenman, J., & Melhuish, C. (2013). “Urinal tryst: how microbial fuel cells can harness urine to generate electricity.” Bioinspiration & Biomimetics, 8(4).
• Walter, X. A., Merino-Jiménez, I., Greenman, J., & Ieropoulos, I. (2016). “Pee power urinal — microbial fuel cell technology field trials in the context of sanitation.” Environmental Science: Water Research & Technology, 2(2), 336–343.
• Bandodkar, A. J., Jeang, W. J., Ghaffari, R., & Rogers, J. A. (2017). “Wearable sensors for biochemical sweat analysis.” Annual Review of Analytical Chemistry, 10, 181–203.
• Yu, Y., Nyein, H. Y. Y., Gao, W., & Javey, A. (2020). “Flexible electrochemical bioelectronics: the rise of in situ bioanalysis.” Advanced Materials, 32(15).
• Kim, S. J., We, J. H., & Cho, B. J. (2014). “A wearable thermoelectric generator fabricated on a glass fabric.” Energy & Environmental Science, 7(6), 1959–1965.
• Leonov, V. (2013). “Thermoelectric energy harvesting of human body heat for wearable sensors.” IEEE Sensors Journal, 13(6), 2284–2291.
• Dagdeviren, C., Yang, B. D., Su, Y., Tran, P. L., Joe, P., Anderson, E., & Rogers, J. A. (2014). “Conformal piezoelectric energy harvesting and storage from motions of the heart, lung, and diaphragm.” Proceedings of the National Academy of Sciences, 111(5), 1927–1932.
• Pavegen (2023). Official Website. Retrieved from: https://www.pavegen.com

Speculative Design | Fart Reactor

1.From Taboo to Technology : Why farts might be worth harnessing.

2.Inside the Fart Reactor : A look at the hypothetical mechanics.

3.Breaking the Silence : Social and cultural impacts of body-powered devices. 👈

4.Beyond Farts : Other human-based bioenergy innovations.

5.Design Challenges : Comfort, efficiency, and privacy concerns.

6.Ethics and Ownership : Who controls the data tied to our bodily by-products?

7.Speculative Futures : Where human-powered tech could lead us next.

1. The Taboo Factor

From Shame to Show-and-Tell?

Flatulence is a near-universal taboo in many societies. The very act is often hidden, joked about, or censored. Turning it into a power source directly challenges that silence. By pushing something “unmentionable” into a public-facing utility, we spark conversations about:

• Body Positivity : Normalizing natural functions rather than viewing them as shameful.
• Environmental Stewardship : Reframing waste as a resource, encouraging sustainable habits at an intimate level.

Cultural Variations

Historically, attitudes aren’t uniform. The 19th-century Japanese He-Gassen scrolls humorously depicted flatulence in art, while French entertainer Joseph Pujol (“Le Pétomane”) made an entire stage career out of controlling and performing with his gas in late 19th-century France. These examples show that what’s taboo in one context can be playful or even celebrated in another.

2. Humour’s Role in Acceptance

Breaking the Ice

Humour is one of the most powerful tools for easing discomfort. By embracing the comedic element of a “fart reactor,” designers and advocates can:

• Disarm Criticism : Acknowledge the silliness upfront, making people more open to hearing the serious benefits.
• Foster Openness : Jokes can transform awkwardness into curiosity, paving the way for deeper discussions about sustainability.

Risks of Trivialization

At the same time, too much humour may undermine the device’s credibility. If it’s perpetually seen as a gag gadget, the real-world potential — whether for micro energy generation or public awareness — might never gain traction. Striking a balance between levity and legitimacy is key.

3. Personal Comfort & Privacy

Bodily Boundaries

A wearable that captures flatulence blurs lines between personal, private functions and public technology. Some users may be fine with it, while others might find it invasive or embarrassing:

• Data Concerns : Could these devices track not just the volume of gas, but also chemical makeup? Who owns this data?
• Medical Insights : On the positive side, gas composition data might reveal information about one’s diet or gut health — but consent and confidentiality matter.

Normalization Over Time

Think of how fitness trackers were once unusual — now, step counts and heart-rate data are openly shared. Similarly, if body-powered devices become mainstream, public sentiment could shift from “That’s so weird!” to “Oh, I see you’re charging your smartwatch.”

4. Social Hierarchies and Identity

Early Adopters vs. Skeptics

If a “fart reactor” ever gained commercial traction, it might be championed by eco-conscious trendsetters. Yet, many would remain hesitant, worried about social ridicule. This dynamic mirrors adoption patterns for many new, stigmatized products — from electric cars once considered “odd” to the widespread use of reusable menstrual products.

Transforming Status

In some speculative futures, a personal biogas device could become a badge of environmental commitment. Much like driving a hybrid car can signal one’s values, wearing a body-powered device might project eco-savviness. Alternatively, it might remain niche, only championed by sustainability hobbyists or outliers comfortable challenging social norms.

It became a norm

5. Looking Ahead

Body-powered devices will likely provoke a broad spectrum of responses — from humour and acceptance to mockery or outright rejection. Yet, by leaning into comedy, emphasizing privacy, and highlighting genuine environmental benefits, designers can carve a path toward mainstream understanding  — if not universal adoption. The more we talk openly about these concepts, the easier it becomes to break down cultural barriers and spark creative thinking about renewable energy at the most personal level.

In the next article, we’ll spotlight other forms of human-generated bioenergy  — from urine to sweat — to see how these “odd” ideas have already emerged in the lab and in limited real-world applications. By expanding our perspective, we’ll better understand how capturing human waste for power is less a wild invention and more a glimpse into what might be possible if we shift our collective mindset.

Speculative Design | Fart Reactor

1.From Taboo to Technology : Why farts might be worth harnessing.

2.Inside the Fart Reactor : A look at the hypothetical mechanics.👈

3.Breaking the Silence : Social and cultural impacts of body-powered devices.

4.Beyond Farts : Other human-based bioenergy innovations.

5.Design Challenges : Comfort, efficiency, and privacy concerns.

6.Ethics and Ownership : Who controls the data tied to our bodily by-products?

7.Speculative Futures : Where human-powered tech could lead us next.

Capturing the Gas

Sealed Intake and Odor Control

• Tight Seal : A specialized undergarment or wearable device would form a snug fit around the body, directing expelled gas into a small collection chamber.
• Odor Filtration : Incorporating a carbon filter or chemical scrubber can neutralize smells. Maintaining wearer comfort and hygiene is paramount for practical use.

Why This Matters
Human flatulence isn’t high in volume or pressure, so capturing every small emission is crucial. A tight seal and efficient odor control ensure minimal leakage while keeping the device discreet.

Concept of a Fart Reactor wearable device

Converting Gas to Electricity

Since the pressure and volume of human flatulence are quite low, traditional mechanical methods (like spinning turbines) are largely impractical. Instead, we look to the following well-researched avenues:

A. Microbial Fuel Cells (MFCs)

How They Work

• Certain bacteria can process organic compounds — including methane or hydrogen sulfide — within a sealed chamber (the “anode”).
• As these microbes break down the gas, electrons are released and travel through an external circuit to the cathode, generating a small but steady flow of electricity.

Pros

• Proven in Lab Settings : Urine-powered MFCs already exist (Ieropoulos et al.). Adapting them for methane-based feedstocks is scientifically plausible. [1]
• Continuous Operation : As long as the bacteria remain active and there’s gas to process, power can be generated.

Cons

• Environmental Control : Bacteria require optimal temperature, pH, and nutrients to thrive.
• Varying Gas Composition : Individual diets can alter the amount and type of gas produced, affecting MFC efficiency.

B. Chemical or Catalytic Conversion

Basic Principle

• Methane can be split or partially oxidized using metal catalysts (e.g., nickel, palladium) to produce hydrogen or directly generate electrons in an electrochemical cell.
• Once hydrogen is formed, a mini fuel cell can convert it into electricity.

Pros

• No Live Organisms : Avoids issues with microbial health or colony maintenance.
• Potentially High Efficiency : Well-established processes at industrial scales, though downsizing is challenging.

Cons

• Engineering Complexity : Reducing large-scale chemical reactors to a wearable format is a major technical hurdle.
• Catalyst Sensitivity : Impurities (such as sulfur compounds in farts) can degrade some catalysts quickly.

Storing and Utilizing the Captured Energy

Even the most efficient conversion will generate small amounts of electricity per emission. This trickle of power can still be useful for:

Micro-Supercapacitors or Mini Batteries

• Quickly store short bursts of electricity from each gas capture event.
• Smooth out inconsistent power generation.

Wearable or IoT Devices

• Power health sensors, LED indicators, or basic Bluetooth connectivity.
• Alleviate some reliance on traditional batteries for ultra-low-power applications.

Realism Check: Is This Feasible?

• Gas Volume : Human flatulence contains some methane, but the total quantity is limited. A fart reactor wouldn’t replace mainstream energy sources — it’s more of a curiosity that underscores how biology can power small electronics.
• Design & Comfort : A device worn daily must be comfortable, discreet, and safe. Filters and collection chambers must be low-profile and easy to maintain.
• Efficiency vs. Cost : Achieving meaningful energy generation may require higher-cost materials or more complex engineering than current consumer wearables.

Nonetheless, exploring these pathways — microbial and chemical conversion — sparks broader questions about how humans might harness any and all forms of waste in a resource-scarce future. Even if a personal fart reactor remains niche or purely conceptual, the underlying technologies have real implications for biogas utilization[2], micro-scale fuel cells, and body-powered innovations.

Speculative Design | Fart Reactor

1.From Taboo to Technology : Why farts might be worth harnessing. 👈

2.Inside the Fart Reactor : A look at the hypothetical mechanics.

3.Breaking the Silence : Social and cultural impacts of body-powered devices.

4.Beyond Farts : Other human-based bioenergy innovations.

5.Design Challenges : Comfort, efficiency, and privacy concerns.

6.Ethics and Ownership : Who controls the data tied to our bodily by-products?

7.Speculative Futures : Where human-powered tech could lead us next.

Challenging a Cultural Taboo

He-Gassen (屁合戦) — A humorous 19th-century Japanese scroll illustrating “fart battles,” showing that some cultures have approached flatulence with comedic artwork rather than strict taboo.

In most contemporary Western societies, flatulence is considered impolite or embarrassing. However, attitudes vary historically and globally. For example, in 19th-century Japan , comedic artwork like the He-Gassen (“Fart Battle”) scroll depicted people using flatulence as a playful weapon. While not a full endorsement of farting in every context, it shows that certain art forms openly joked about or depicted flatulence without the same level of taboo. Similarly, Le Pétomane (Joseph Pujol) became a famous French stage performer in the late 19th century by entertaining audiences with his “art” of controlled flatulence — a reminder that, in some comedic traditions, passing gas could be a central attraction rather than a forbidden topic.

These examples don’t necessarily mean entire cultures fully “approve” of public farting, but they demonstrate that the strict hush-hush treatment so common in many parts of the world isn’t universal. Speculative design  — a practice that encourages us to see everyday objects and systems in radically new ways — prods us to imagine: what if we looked at our own bodies’ by-products as a resource rather than an inconvenience?

By reframing something culturally off-limits into a design challenge, we spark conversations on broader themes: the value of human waste, the quest for eco-friendly solutions, and the willingness of society to embrace unconventional ideas. Speculative design takes note of these pockets of acceptance (or at least humour) and asks: if we can talk about it, maybe we can harness it.

Hidden Energy Potential

Despite its comedic reputation, a fart is a small but tangible emission of gas. It often contains methane, hydrogen, carbon dioxide, and other compounds that — if collected in large quantities — can be combusted or used in chemical reactions to produce electricity. Indeed, capturing methane is standard practice on farms and in landfills, where organic matter decomposes to form biogas.

Ieropoulos, I. A., Greenman, J., & Melhuish, C. (2013). “Urinal tryst: how microbial fuel cells can harness urine to generate electricity.” Bioinspiration & Biomimetics, 8(4).

Shifting that large-scale approach to personal flatulence might seem outlandish, but speculative design invites us to consider how tiny individual contributions, taken together, could power small electronics or sensors. The crucial idea is transforming bodily waste from an object of laughter or shame into a resource that fits within larger sustainability goals.

In Dune, Frank Herbert envisions a harsh desert world where survival hinges on conserving every drop of moisture. The Fremen — inhabitants of the sandy planet Arrakis — rely on “stillsuits,” specialized desert overalls designed to reclaim bodily fluids like sweat and exhaled vapour. These suits are a fusion of practicality and ingenuity, capturing and filtering moisture so that even in an arid, life-threatening environment, its wearers can remain hydrated. The stillsuits not only reflect the Fremen’s deep respect for water as a precious resource but also emphasize the broader theme of ecological balance that runs throughout the story.

Herbert, F. (1965). Dune. Chilton Books. (Concept of stillsuits as moisture-recycling garments.)

This idea parallels how speculative designs might one day harness other bodily by-products, including flatulence, to create energy in resource-limited environments. Much like the stillsuit’s water-recycling function, a “fart reactor” concept seeks to transform what we ordinarily dismiss as waste into a vital, sustainable resource. Both examples illustrate how necessity and ingenuity can merge to reshape our relationship with our own bodies’ outputs — and potentially redefine survival in extreme conditions.

Why Now?

Human-generated biogas is only one facet of the growing field of waste-to-energy technology. Researchers have long harnessed methane from livestock manure, while engineers experiment with microbial fuel cells powered by human urine. The intensifying push for green solutions has also led to innovations like piezoelectric flooring (harvesting energy from footsteps) and wearable thermoelectric generators (converting body heat into power).

In that context, a so-called “fart reactor” is one more step toward reevaluating what we consider “waste.” If it can reduce even a fraction of our reliance on fossil fuels — or spark bigger, bolder energy ideas — then perhaps it’s worth pushing against cultural taboos.

Looking Ahead

Over the next articles, we’ll explore this concept from multiple angles — from how a wearable “fart reactor” might technically function to what happens if it’s widely adopted. We’ll dive into social etiquette (could it ever be normalized?), privacy concerns (what data gets captured?), and ethical questions (who truly benefits from harnessing human-produced gas?). We’ll also examine existing and emerging waste-to-energy solutions that prove, in some cultures and contexts, bodily by-products can be a legitimate power source.

Stay tuned  — as you’ll see, there’s more to this idea than meets the nose. By confronting taboos and embracing the unexpected, we open ourselves to new possibilities for environmental stewardship and inventive design.

In December, I decided to take up drawing as a hobby. I enrolled in a few basic online courses and started sketching. One evening, my daughter saw me drawing and said, ‘I want to try that too!’ And that’s how we started having 30–40 minute evening drawing sessions.

A few months ago, during one of those sessions, my daughter sat down with a drawing pad, determined to create her best masterpiece. She picked up her pencil, drew a few lines, and then sighed in frustration. ‘It’s ugly,’ she said, crumpling the paper and tossing it aside. ‘I can’t draw!’

I picked up the paper, smoothed it out in front of her, saying, ‘You know, even great artists had to start somewhere. You don’t have to get it perfect — you just have to keep going.’

I showed her how to approach drawing: First, you draw a light outline, without pushing the pencil hard. This establishes the scale and proportions. It’s still a draft, but it starts reflecting the composition, and at this point, you may decide what to correct, what to drop, or even discontinue. Then, you add main contours and shadows. It’s still a draft, but it starts reflecting the reality. Finally, you add finishing touches, but this process can go on forever — it’s up to the artist to decide when to stop, when the draft is not a draft anymore.

I am very proud of her. She started playing with shades, and her drawings are becoming more and more interesting. Indeed, children are more artistic than adults.

One of my daughter drawings

This process mirrors so much of what we do in life and work. We want to start perfect. We get frustrated when things don’t go the way we expect. But the truth is — whether it’s drawing, starting a business, or learning a new skill —  Version 1.0 is never perfect.

A while back, two of my friends came to me with an idea. They had spotted a market gap — a real opportunity! We were excited. We brainstormed, planned, and worked late nights. But instead of shipping a minimal version of our product quickly , we spent months perfecting it — every feature, every detail. We were so focused on getting everything right before launching that we burned through time, energy, and most importantly, our budget.

Finally, after nearly a year of refining, I realized we needed to launch. We had accumulated enough features that we could finally show them to real users. My cofounders reluctantly agreed, still wanting the product to be perfect. But we pushed ahead. To our surprise, the first week went well  — we sold a few subscriptions. But then, the real feedback started rolling in…

As users began using the product, we encountered bugs, data misalignments, and broken workflows. But the worst part was that many of our assumptions were wrong. If we had launched earlier, we could have tested our ideas, adjusted our approach, and saved ourselves a lot of wasted time and resources.

Looking back, the biggest lesson I learned was: fail early, adapt quickly. Perfection doesn’t happen overnight. Whether it’s drawing or launching a product, the key is to start and improve along the way. If we wait for perfection, we get stuck. But if we start, learn, and improve — we grow.

So, just like with drawing, the most important step is not to get it perfect but to keep going, keep learning, and keep iterating. Version 1.0 is never perfect… but it’s the only way to get to Version 2.0.

I miserably failed drawing a human body…The proportions are wrong. But this is Version 1.0

Picture this :
You step into a softly lit coffee shop. It looks like any modern café: sleek tables, a warm wood interior, and people quietly immersed in their laptops or conversations. Behind the counter, a subtle sign reads:
“100% Off-Grid — Powered by Collective Biogas.”

At first glance, nothing seems unusual. There’s no noticeable noise beyond the gentle hum of voices and espresso machines. The air is clear and free of any hint of odor. A few customers wear small, sleek patches on their lower back, but they look more like typical fitness trackers than anything else. No one seems to pay them much attention.

Near the charging station, there’s a discreet display showing a rolling energy graph. Every so often, a faint blue bar shifts upward — an almost imperceptible nod to the moment when someone’s device quietly captures and converts a tiny amount of biogas into electricity. Yet there’s no fanfare or comedic flash. No curious odors, no sounds. It’s all seamless and routine.

Patrons come and go, plugging in their phones and sipping lattes as data streams across the display. People appreciate the café’s sustainable mission, but they don’t dwell on the mechanics. The technology has faded into the background — a quiet, invisible layer of everyday life.

Welcome to Our Speculative Design Series on the “Fart Reactor”!

Concept of a Fart Reactor wearable device

This seven-part exploration turns a playful idea — capturing flatulence for clean energy — into a springboard for rethinking how we harness human biowaste. Using speculative design , we challenge taboos and pose fresh questions about technology, culture, and sustainability.

Here’s what’s coming up:

1. From Taboo to Technology : Why farts might be worth harnessing.
2. Inside the Fart Reactor : A look at the hypothetical mechanics.
3. Breaking the Silence : Social and cultural impacts of body-powered devices.
4. Beyond Farts : Other human-based bioenergy innovations.
5. Design Challenges : Comfort, efficiency, and privacy concerns.
6. Ethics and Ownership : Who controls the data tied to our bodily by-products?
7. Speculative Futures : Where human-powered tech could lead us next.

Seem far-fetched? Join us as we dive deeper into how this oddball concept of a “fart reactor” could transform not only our perceptions of human waste, but also our approach to sustainable design and the future of everyday living.

At first, this might seem complex, but the core ideas are straightforward. I’ll break them down using simple examples and clear explanations, so no technical background is needed.

Take a popular messaging app on your phone as an example. When you send a message, it travels through a series of network devices — like routers and switches — before reaching a server. This server stores, processes, and manages your message. To ensure reliability, the message is often replicated across multiple servers, creating backups. This redundancy ensures your message is safe; even if one server fails, you can still access your data seamlessly. The illustration below shows how this process works.

In a sequence diagram, each actor is not tied to a fixed number. The message-sending process could involve 10 devices — or 100. Viewed from another angle, the complexity of this flow becomes evident.

However, the brilliance of technology lies in its ability to hide this complexity. The user interface simplifies everything, allowing us to send and receive data effortlessly without seeing the intricate processes behind the scenes. Today, transferring gigabytes [3] of data is seamless, and as data usage grows exponentially, our world becomes increasingly data-driven and intelligent.

It’s often said that “data is the oil of modern life,” but there’s another, often-overlooked factor: electricity. I like to think of electricity as the oxygen that powers all these technologies. Energy-intensive innovations like AI, blockchain, and electric vehicles (EVs) are driving electricity demand, which can lead to shortages and, more importantly, significant environmental impacts.

Recent analyses [5] indicate that storing 1 GB of data in the cloud consumes approximately 0.1 kilowatt-hours (kWh) of electricity per year. This figure accounts for significant improvements in data center energy efficiency compared to earlier estimates, which ranged from 3 to 7 kWh per GB annually.

To put this into perspective, a typical household in the UK consumes about 2,700 kWh of electricity annually. Therefore, storing 1 terabyte (TB) of data in the cloud would account for approximately 100 kWh per year, representing around 3.7% of the average household’s annual electricity consumption.

It’s important to note that these figures can vary based on factors such as the energy efficiency of specific data centers and the local electricity mix. Nonetheless, as our reliance on cloud storage grows, so does its cumulative environmental impact.

Beyond electricity usage, data centers generate considerable digital waste, such as worn-out or broken hardware components. Hard drives, for example, have a limited lifespan and must be replaced after a certain period, regardless of their condition.

The more data we generate and store, the more energy we consume, and the larger our environmental footprint grows.

Recently, I came across a research paper discussing the ethical use of online services. It sparked my curiosity and led me to explore the topic further. To my surprise, despite the growing importance of this issue, it remains largely overlooked. This is my small contribution to shedding light on it.

So, what can we do? You might wonder.

On a personal level, we can focus on reducing energy consumption — using energy-efficient appliances and being mindful of electricity usage at home. But does this mean we need to abandon modern technology and convenient online services? Not at all. Instead, we should strive to use them more consciously.

For instance, many of us unknowingly have “zombie resources” — services [4] or accounts we once used but abandoned long ago. These inactive resources still consume energy and occupy space. Take a moment to identify and clean them up.

Another growing trend is AI prompting, which is becoming a popular way to search for information. However, AI systems consume significantly more energy than traditional searches — often 10 to 100 times more [1] [2]. If the query is trivial, consider using a standard search engine instead.

Additionally, not everything needs to be stored in the cloud. Local storage is thousands of times more energy-efficient and can be powered down when not in use. Similarly, when it comes to communication, text messages are far smaller in size compared to video or audio messages. If the message can be text-based, consider typing or using your phone’s speech-to-text feature.

As we’ve seen, every digital action — no matter how seamless — burns electricity at every step of its journey. While technology continues to evolve, we must recognize that energy efficiency in modern tools is still a work in progress.

Until more sustainable innovations are developed, it’s on us to be mindful of our energy usage. Behind every sleek and engaging user interface lies hidden resource consumption. By being conscious of this, we can reduce our environmental impact while continuing to enjoy the benefits of the digital age.

References

[1] Energy Consumption of AI Searches vs. Traditional Searches: AI-powered search engines, such as those utilizing GPT-4, consume significantly more energy due to the computational power required. A single AI search query can consume up to 10 times more energy than a traditional search.

[2] Energy Consumption of Data Centers: Data centers, essential for AI training, account for almost 1% of the world’s energy consumption. This figure is set to rise over the next few years.

[3] Energy Consumption of Data Transmission: The energy required to transmit data over the internet varies, with estimates suggesting that transferring 1 GB of data can consume between 0.0064 kWh to 136 kWh, depending on the method and distance of transmission.

[4] Energy Consumption of Cloud Services: While employees may not think about the emissions associated with cloud services, the energy consumption and emissions generated through their use of email, cloud storage, and collaboration technologies still contribute to their company’s scope 3 emissions.

[5] Energy Consumption of Cloud Storage: Storing data in the cloud requires energy for data transmission and storage. Estimates suggest that storing 1 GB of data in the cloud consumes approximately 0.1 kWh per year.

A good desk might be built by a dilettante by simply assembling different parts.

First, you’ll need to gather your materials. You’ll need a worktop, legs, screws, and a drill. You can find these materials at your local hardware store or online. My selection is very budget-friendly and easy to find in Finland, but you can choose any materials you like.

• Worktop: I chose a 2000mm x 600mm x 26mm glued timber panel acacia oiled from Bauhaus. Cost: 89 EUR.

Desk top brand

Texture of top

The appealing part is that you may find a texture that suits best for you.

• Legs (2x): I chose an MITTBACK adjustable leg from IKEA. Cost: 40 EUR x 2 = 80 EUR.

MITTBACK leg

Though I should have chosen a cheaper option, I liked the design and the adjustable height. However, I could have saved at least 20 EUR by choosing a cheaper option.

Once you have your materials, you can start building your work desk. The first step is to attach the legs to the worktop. You can do this by marking hole places (pencil can help) and after that drilling pilot holes in the worktop and then screwing the legs into place.

Pilot holes

Make sure the legs are evenly spaced and secure.

Assembled desk

Once the legs are attached, you can add any finishing touches, such as a coat of paint or varnish. And that’s it! You now have a beautiful work desk that you built yourself. I have done nothing to my worktop.

Desk

I’m very happy with the result. The desk is sturdy, looks great, and fits perfectly in my home office. I also saved a lot of money by building it myself. The total cost was 169 EUR, which is a great deal for a desk of this.

I hope this post has inspired you to build your own work desk. It’s a fun and rewarding project that anyone can do. Happy building!

1. Memory allocation: When a program needs to store data, it requests memory from the operating system. The memory can be allocated statically, at compile time, or dynamically, at runtime.
2. Memory deallocation: Once the program is finished using the memory, it must be returned to the operating system. Failure to do so can lead to memory leaks, which can cause the program to crash or slow down over time.
3. Garbage collection: In some programming languages, such as Java and Python, memory management is automated through a process called garbage collection. This involves periodically scanning the program’s memory to identify and free up memory that is no longer in use.
4. Pointers: In low-level programming languages such as C and C++, memory is managed using pointers. These are variables that store memory addresses, allowing the program to manipulate memory directly. However, incorrect use of pointers can lead to memory leaks and other errors.
5. Memory safety: Memory safety refers to the prevention of memory-related errors such as buffer overflows, null pointer dereferences, and use-after-free errors. Programming languages that prioritize memory safety, such as Rust and Swift, use techniques such as ownership and borrow checking to ensure that memory is used correctly.

What is the main idea behind garbage collection? How does it work?

The main idea behind garbage collection is to automate the process of memory management in programming languages by automatically detecting and freeing memory that is no longer in use. This eliminates the need for programmers to manually allocate and deallocate memory, which can be error-prone and time-consuming.

Garbage collection works by periodically scanning the program’s memory to identify memory that is no longer in use. This is typically done by tracing through the program’s data structures to find all the objects that are still reachable from active variables or from the stack. Any memory that is not reachable, or that is only reachable through other unreachable memory, is considered garbage and can be freed.

The garbage collection algorithm typically uses a combination of reference counting and tracing to determine which memory is still in use. Reference counting involves keeping track of the number of references to each object, and freeing the object when the reference count drops to zero. Tracing involves starting from a set of “roots” (typically active variables and the stack) and following all the pointers to other objects in memory. Any object that is not reached during the tracing process is considered garbage.

There are various algorithms that can be used for garbage collection, such as mark-and-sweep, generational, and concurrent garbage collection. Each algorithm has its own trade-offs in terms of performance, memory usage, and responsiveness, and the choice of algorithm depends on the specific requirements of the programming language and the application being developed.

What is the main idea behind pointers? How does it work?

Do you know what to do with pointers? 😉

The main idea behind pointers in programming is to allow direct manipulation and referencing of memory locations in a program. A pointer is a variable that stores the memory address of another variable, allowing the programmer to access and modify the contents of that memory location.

When a pointer variable is declared, it is typically initialized with the memory address of another variable using the “address-of” operator (&). For example, the following code declares a pointer variable p and initializes it with the address of the integer variable x:

int x = 42;
int *p = &x;

The * in the declaration of p indicates that p is a pointer to an integer.

Once a pointer variable is initialized, it can be used to access and modify the value of the variable it points to using the “dereference” operator (*). For example, the following code modifies the value of x indirectly by dereferencing p and assigning a new value:

*p = 123;

After executing this code, the value of x will be changed to 123.

Pointers are particularly useful in low-level programming languages such as C and C++ for tasks such as memory management, data structures, and system-level programming. However, they can be error-prone if used incorrectly, as they allow the programmer to manipulate memory directly, which can lead to memory leaks, buffer overflows, and other types of memory-related errors.

What is the main idea behind memory safety? How does it work?

The main idea behind memory safety in programming is to prevent errors related to memory management, such as buffer overflows, null pointer dereferences, use-after-free errors, and other security vulnerabilities. Memory safety ensures that programs do not access memory in a way that is unintended or unsafe, which can cause crashes or allow attackers to exploit vulnerabilities.

Memory safety works by enforcing strict rules on how memory is accessed and managed by a program. These rules are typically enforced by the programming language or the runtime environment, and can include the following:

1. Bounds checking: Memory safety mechanisms can check that array and buffer accesses are within their allocated bounds, preventing buffer overflows and other memory-related errors.
2. Null pointer checks: Memory safety mechanisms can prevent null pointer dereferences by checking that pointers are not null before they are dereferenced.
3. Ownership and borrowing: Memory safety mechanisms such as those in the Rust programming language can enforce ownership and borrowing rules to ensure that memory is only accessed in a safe and controlled manner.
4. Garbage collection: Garbage collection, as mentioned earlier, is a memory safety mechanism that automatically manages memory by freeing objects that are no longer in use.
5. Memory-safe programming languages: Certain programming languages such as Rust, Swift, and Java are designed with memory safety in mind and enforce strict rules on how memory is managed and accessed by a program.

By enforcing these and other memory safety mechanisms, programming languages and runtime environments can help ensure that programs are safe and secure, and prevent a wide range of memory-related errors and vulnerabilities.

What programming languages use generational garbage collection techniques for managing memory?

Generational garbage collection is a technique used in many programming languages for managing memory. Some of the programming languages that use generational garbage collection include:

1. Java: Java’s garbage collector uses a generational algorithm to manage memory. The heap is divided into young and old generations, and objects are initially allocated in the young generation. The garbage collector uses a mark-and-sweep algorithm to identify and free unused objects in the young generation, and periodically moves surviving objects to the old generation.
2. Python: Python’s garbage collector uses a variant of the generational algorithm called “incremental garbage collection”. The heap is divided into three generations, and objects are initially allocated in the youngest generation. The garbage collector uses a combination of reference counting and tracing to identify and free unused objects in the young generation, and periodically moves surviving objects to older generations.
3. Ruby: Ruby’s garbage collector uses a generational algorithm with two generations. Objects are initially allocated in the young generation, and the garbage collector uses a mark-and-sweep algorithm to identify and free unused objects in the young generation. Surviving objects are moved to the old generation, which is managed using a separate garbage collector.
4. Go: Go’s garbage collector uses a tri-color mark-and-sweep algorithm with two generations. Objects are initially allocated in the young generation, and the garbage collector uses a mark-and-sweep algorithm to identify and free unused objects in the young generation. Surviving objects are moved to the old generation, which is managed using a separate garbage collector.
5. JavaScript: Many JavaScript engines, including V8 and SpiderMonkey, use a generational garbage collector with two or three generations. Objects are initially allocated in the youngest generation, and the garbage collector uses a mark-and-sweep algorithm to identify and free unused objects in the young generation. Surviving objects are moved to older generations, which are managed using separate garbage collectors.

These are just a few examples of programming languages that use generational garbage collection to manage memory. Many other programming languages also use this technique, or variations thereof, as it is a popular and effective way to manage memory in modern programming languages.

Is the garbage collection process blocking operation?

Whether or not the garbage collection process is a blocking operation depends on the implementation of the garbage collector and the programming language or runtime environment in use.

In some cases, garbage collection can be a blocking operation, meaning that the program must pause execution while the garbage collector runs. During this time, the program cannot perform any other operations, which can result in decreased performance or even temporary freezing of the program. This is often the case with “stop-the-world” garbage collectors, which pause the program during garbage collection.

However, not all garbage collectors are blocking. Some garbage collectors use concurrent or incremental algorithms, which allow the program to continue executing while garbage collection is performed in the background. These algorithms are designed to minimize or eliminate the pauses that occur during garbage collection, which can improve program performance and user experience.

Some programming languages, such as Java and .NET, use a combination of different garbage collection algorithms, including both blocking and non-blocking algorithms. These languages also provide options for configuring garbage collection behaviour, which can allow developers to tune the performance of the garbage collector to meet their specific needs.

In summary, whether or not garbage collection is a blocking operation depends on the implementation of the garbage collector and the programming language or runtime environment in use. While some garbage collectors may be blocking, others use non-blocking or concurrent algorithms that allow the program to continue executing while garbage collection is performed in the background.

What is an incremental garbage collector?

An incremental garbage collector is a type of garbage collector that allows a program to continue executing while garbage collection is performed in the background. This is in contrast to “stop-the-world” garbage collectors, which pause the program during garbage collection.

In an incremental garbage collector, the garbage collector breaks the garbage collection process into small, incremental steps that are interleaved with the program’s normal execution. For example, the garbage collector might perform one step of garbage collection for every 10 or 100 operations executed by the program. By breaking the garbage collection process into smaller steps, the garbage collector can minimize the impact on program performance and reduce the likelihood of noticeable pauses or freezes.

Incremental garbage collectors are typically used in environments where responsive performance is critical, such as interactive applications or real-time systems. By allowing the program to continue executing during garbage collection, these systems can maintain a more consistent level of responsiveness and avoid long pauses that could negatively impact user experience.

One disadvantage of incremental garbage collection is that it can increase the overhead of garbage collection, as the garbage collector must maintain more state and perform more bookkeeping to track the progress of garbage collection. This can result in slightly slower garbage collection performance compared to stop-the-world garbage collectors. However, the benefits of maintaining program responsiveness often outweigh this overhead.

Many programming languages and runtime environments support incremental garbage collection, including Java, .NET, and some JavaScript engines.

Trilemma: throughput, pauses, resources (CPU/Memory)?

The garbage collection trilemma refers to the challenge of balancing three competing goals in garbage collection: throughput, pauses, and resource utilization (CPU and memory usage).

1. Throughput refers to the rate at which the garbage collector can identify and free unused memory. To maximize throughput, the garbage collector should be able to quickly identify unused memory and efficiently free it.
2. Pauses refer to the amount of time that the program is paused while garbage collection is performed. To minimize pauses, the garbage collector should be able to identify and free unused memory quickly and efficiently without causing significant delays to the program execution.
3. Resource utilization refers to the amount of CPU and memory resources used by the garbage collector. To minimize resource utilization, the garbage collector should be designed to use resources efficiently and effectively, without consuming too much CPU or memory resources that could be used by the program.

Managing this trilemma requires a careful balance of these competing goals. Different programming languages and runtime environments may use different garbage collection strategies to manage the trilemma, depending on the specific needs of the application and the available system resources. Some programming languages and environments allow adjusting a garbage collector’s behaviour.

Tuning garbage collection to balance throughput, pauses, and resource utilization depends on the specific garbage collection algorithm being used and the requirements of the application being developed. Here are some general strategies that can be used to tune garbage collection:

1. Throughput: To maximize throughput, a generational garbage collection algorithm can be used. This algorithm focuses on the youngest generation of objects, which are typically short-lived and can be freed quickly. Additionally, the garbage collector can be tuned to allocate more memory to the young generation, which can reduce the frequency of garbage collection cycles and improve throughput. Other strategies for improving throughput include parallelizing garbage collection across multiple processors and using an incremental or concurrent garbage collector that runs in the background while the program is executing.
2. Pauses: To minimize pauses, a concurrent or incremental garbage collector can be used. This garbage collection algorithm allows the program to continue executing while garbage collection is performed in the background, minimizing the impact on program performance. Additionally, a “pauseless” garbage collector can be used that is able to free memory without stopping the program’s execution. Other strategies for reducing pauses include optimizing the garbage collector to reduce the frequency and duration of garbage collection cycles and using a garbage collector that is able to dynamically adjust its behaviour based on the available system resources.
3. Resource Utilization: To minimize resource utilization, a “greedy” garbage collector can be used that aggressively frees memory and minimizes the amount of memory used by the program. Additionally, the garbage collector can be optimized to reduce the amount of CPU and memory resources required to perform garbage collection, such as by using efficient data structures and algorithms. Other strategies for reducing resource utilization include using a garbage collector that is able to dynamically adjust its behaviour based on the available system resources and tuning the garbage collector to allocate memory more efficiently.

Overall, tuning garbage collection requires a careful balance of throughput, pauses, and resource utilization. Different programming languages and runtime environments may use different strategies to manage the trilemma, depending on the specific needs of the application and the available system resources.

What can an application achieve by tuning the garbage collector?

Tuning the garbage collector can help applications achieve several benefits, such as:

1. Improved Performance: Tuning the garbage collector can lead to improved application performance by reducing the amount of time spent on garbage collection and minimizing the impact of garbage collection on application response time.
2. Reduced Memory Footprint: Garbage collection tuning can help reduce the amount of memory consumed by the application, leading to reduced memory usage and improved scalability. This can be especially important in cloud environments where memory usage can have a direct impact on application cost.
3. Reduced Latency: Tuning the garbage collector can help reduce the amount of time spent on garbage collection pauses, leading to lower application latency and improved user experience.

Some cloud service-related examples of tuning the garbage collector include:

1. AWS Lambda: AWS Lambda allows users to configure the amount of memory allocated to each function instance. Tuning the garbage collector can help reduce memory usage and improve the performance of Lambda functions, which can lead to reduced costs and improved application scalability.
2. Azure Functions: Azure Functions also allows users to configure the amount of memory allocated to each function instance. Garbage collection tuning can help reduce the impact of garbage collection on function response time and improve application performance.
3. Google Cloud Functions: Google Cloud Functions automatically scales the amount of memory allocated to each function instance based on the workload. Tuning the garbage collector can help ensure that garbage collection does not consume too much CPU or memory resources, leading to improved performance and lower costs.

Overall, tuning the garbage collector can help improve application performance, reduce memory footprint, and improve user experience in cloud environments.

In summary, having knowledge of data structures and memory management is essential for writing efficient and performant code in any programming language. By understanding how memory works in a particular language, developers can make informed decisions about how to allocate and manage memory resources for their applications.

Knowing the specifics of a language’s memory management can also help developers optimize their code for performance and avoid common memory-related issues, such as memory leaks, dangling pointers, and buffer overflows.

Moreover, understanding memory management is particularly important in the context of modern computing, where applications are often designed to run in cloud environments with limited resources. In these environments, optimizing memory usage can help reduce costs and improve application scalability.

Overall, having memory awareness and an understanding of how data structures work and how memory is managed in a particular language is critical for building efficient, scalable, and robust applications. It can help developers avoid memory-related issues, optimize performance, and improve user experience in a variety of computing environments.

Particularly, I want to cover the following questions:

• How to observe the system’s landscape that is made up of N projects?
• How to build complex checks that may evaluate some specific cases?
• How to scale sanity checks and run them on a regular base?
• How to share a system design with different audiences?
• How to examine critical/crucial areas in a code that are important from an architectural point of view?

By addressing these questions, I want to share my set of tools though I will not dig into great details because it might be super boring, in fact not😀, and time-consuming. Rather than that, I am going to give some hints and ideas for further independent investigations and learning.

How to observe the system’s landscape that is made up of N projects?

Practically, the majority of developers are familiar with a dependency graph where a person can observe what dependencies are in a particular project. Likewise, well-designed solutions follow architectural patterns, principles and rules, which define how to separate layers and modules, establish communications, and so on. Analyzing dependency graphs may detect possible architectural problems at earlier stages. Although there are many tools for visualizing graphs, not all of them are handy. For example, diagrams that are generated by inbuilt Visual Studio or Rider are superficial, as a result, they do not give much information. Nevertheless, there is a tool that covers my needs, and its name is NDepend. The prime advantage of it over other tools is that produced graphs are dynamic, where by dynamic I mean that a dev can manipulate the graph by making custom queries and filterings, clustering components to simplify the diagram, visualizing the dependencies’ paths and cyclings, searching and highlighting, and many other things. I reckon that this part deserves its separate article, and there is an official page that shows the basics, which I highly recommend reading if you want to start using it. Moreover, a user can go throughout assemblies, namespaces, classes, and even methods and fields, which gives an incredible level of granularity, where by double-clicking on a diagram element, the user may jump to the lines of corresponding code. Therefore, any of us can observe the structure of a big solution in great detail and be super-efficient. The picture below illustrates how it looks in Visual Studio (Note: NDepend has a standalone application, and there are extensions for Visual Studio and Azure DevOps).

I am sorry for the readability of the diagram, it was taken from an actual project, and I tried to make it ‘anonymous’. However, I wanted to show how it can cluster projects, and there are various relationship types, which are shown on the right-side panel. To get a better grasp I recommend watching this 6 minutes video.

NDepend — Dependency Graph

How to build complex checks that may evaluate some specific cases?

Another very powerful feature of NDepend is its query language CQLinq, which allows querying the code with C# and LINQ. Basically, it doesn’t have a steep learning curve, every C#/.NET developer should easily start leveraging it. With that people are capable to build their own specific requirements for the code via creating queries, rules, and quality gates. For example, imagine that our team was practising DDD concepts, and we wanted to be sure that in an application, aggregates are light, that they have a depth of fewer than 3 objects. If that rule was violated, NDepend would show a warning message to indicate the issue. Then the team would consider refactoring that aggregate to prevent it from bloating and smelling. An example below shows how to implement a simple WARN rule in CQLinq that notifies if a type name that ends with the “Agg” suffix and its cyclomatic complexity is more than 15 (Note: we assume that a team agreed to mark all aggregates within an application with “Agg” suffix).

NDepend — Custom rule in CQLinq

As we can see from the code example above, the syntax is simple and doesn’t require much effort from devs. Consequently, devs have a solid tool that opens the possibility to build comprehensive checkers of any complexity.

Moreover, NDepend logic is prevalently based on Rules and Quality Gates. The difference between those is that the first one helps to keep code cleaner, and prevents it from bashing and potential problems, while the second one outputs one of the statuses (Pass, Warn, Fail) and could be considered as a quality gate for shipping software to production. NDepend has a very good pack of pre-installed sets of rules and quality gates where some of them can be turned off/on. And of course, a developer can extend the list of rules/quality gates by adding his/her own custom rule/quality gate.

How to scale sanity checks and run them on a regular base?

Controlling the code quality is an important part of any development process, it requires a significant amount of seniority level from its participants and the most valuable thing is time. Nevertheless, sometimes due to constraints and time pressure we may omit this part or do it superficially. Although some basic quality checks can be verified by static code analyzing tools and even be extended by custom directives, unfortunately, it is still not enough. What if we can combine the power of CQLinq, run its rules on a regular base via CICD and produce a report where you can see the quality of the code-base in a convenient graphical view. Indeed, we can do that, there is an assembly “/net5.0/NDepend.Console.MultiOS.dll” that could be executed on Linux machines via terminal. Accordingly, if we can execute it in the terminal, we can do that in CICD pipelines, and here is the link to how to do that with a more profound description. The resultant of it would be a report, which comprises HTML and Javascript files.

NDepend — generated report

First of all, the report is sort of interactive, so that, a specialist is able to observe many different diagrams, charts, metrics, etc. By navigating, clicking links, and hovering the mouse on some types of elements, the specialist may get concise information about the state of the system. The picture below illustrates the Abstractness vs. Instability diagram that shows which assembly could be potentially difficult to maintain. Dots represent assemblies, by hovering the mouse it is possible to see more details.

NDepend — Abstractness vs. Instability diagram

In addition, the report provides a technical debt estimation in human hours format with a level of severity. This might be useful for risk analysis, or simply for planning the debt compensation work. However, this is just an estimation which is made by an algorithm, so, it shouldn’t reflect reality. Rather than that, it might be good to consider it as a complementary indicator that shows the complexity of a particular issue.

Unfortunately, the world is not a perfect place, and NDepend like any other software product has its pros and cons. I encountered a lot of pros in the first part of the article, so, let’s continue with the cons. The first drawback is that on a Windows machine, the experience is much richer. For example, there is an extension for Visual Studio that provides a simple GUI, furthermore, it is also possible to run it as a standalone application. However, it is not the same for macOS or Linux OSs, there is only the MultiOss.dll file that should be executed in a console. I was able to mitigate this problem by installing Parallels Desktop software on macOS, where I can use windows apps along with native applications, but it is an extra expenditure. The second con is it costs money, and it is not cheap. I guess they are able to achieve more popularity at the cost of lowering the price. However, NDepend offers 14 trial days for hands-on experience, though it might not be enough to get a grasp of the product (they could extend the trial to 30 days).

How to share a system design with different audiences?

As usual, various groups are eager to pursue their interests, and all of them have different backgrounds and knowledge. Based on these facts, we can target the information to a specific group of people and convey our ideas and design more concisely. There are different methods how to do that, but I found one very simple method. It is the C4 model for visualizing software architecture by Simon Brown. The C4 represents four levels of abstractions, such as Context, Container, Component and Code where each type has its semantic payload. The picture below depicts the hierarchy of these diagrams.

C4–4 levels of the C4 model

• The first diagram is the Context Diagram that shows the big picture, how the system fits in an existing world, what it does, and its relationships with other external systems and users. This type of diagram could be shown to a broad audience including non-technical people, and it can be shared with external participants.
• In the same way, Container Diagrams depict the system’s internals, specifically how container-wise it is organized, where a container is a self-deployable unit ( don’t confuse it with a Docker container). For example, if the system consists of a database, backend, and client applications, the diagram will illustrate 3 containers and their relationships within a system border, moreover, each container will have a brief description and tech stack. This type of diagram could be shown to an audience that has a common understanding of how information systems work, though it also contains some technical details. And it can be shared with external stakeholders as well.
• In comparison with the aforementioned diagrams, the Components Diagram shows components and their interactions in a particular container where each component should describe its purpose and technical details. The audience is technical people, and it reveals a lot of internals, hence, it is not recommended to share it with external people. This diagram is optional.
• And the last diagram, the Code Diagram shows how a particular part of a system was built in terms of interfaces, classes and derivations. This diagram is optional. Furthermore, the C4 suggests not to design it because abstractions on a code level have a very high pace of changes, and it is difficult to keep up with updating the diagram. Likewise, the code-level diagrams could be generated with a help of modern IDEs.

The notation of the C4 model is not described here intentionally because they are pretty simple and the rules can be obtained from the documentation. Last but not least, I want to propose 3 tools that I use for drawing C4 diagrams:

• draw.io — visual diagramming tool. Not the most efficient one, but it is useful when the diagram is getting complex and you need to adjust some parts specifically, and maybe, add some additional notations.
• plantUML — text-based diagramming tool. This tool is in the golden middle because it is simple enough and powerful at the same time. With a help of an extension from GitHub, the diagrams look very good. The only shortcoming is when the diagram is getting bigger, it might be difficult to tell plantUML how to draw some particular areas in order to increase the readability. Although, I found that in 90% of cases, it works well for Context and Containers diagrams.
• Structurizr — it is a text-based modelling tool, the most powerful and expensive 💰. It grants one free workspace, and it can be enough for your own ‘pet’ project or just to try it out. Nevertheless, one workspace is not suitable for a real project.

How to examine critical/crucial areas in a code that are important from an architectural point of view? (very quick glance 🐎)

From time to time, I have to write high performant code for some specific areas. To know what has been affected by my changes, I need to test and measure it. In fact, there is a vast range of diagnosing tools that can be used, but I want to show what I have been using for years. The very first is an amazing BenchmarkDotNet .NET library that does all the heavy work when you do micro benchmarking (testing classes and methods). The library may help to optimize some subtle things. Besides that, for a very quick investigation, there is another tool that can help to evaluate the .NET code and LINQ queries as well, by presenting the intermediate results of code compilation, its name is LINQPad. It has GUI but it is only a Windows tool, which is a downside of it. And quite recently, I discovered for myself a very useful .NET playground website SharpLab by Andrey Shchekin. Although it doesn’t measure the performance of a code, it is a great resource for learning the internals of .NET and new language features. In the example below, we do an inspection of memory graphs of a newly created array of strings and array of chars. On the right side, we can see how they are allocated in memory, what is on a stack or heap, and their references. As you see, it is a very explanatory visualization of the complex memory management aspect.

SharpLab — Inspection of a memory graph

Apart from these appliances, there is also a good set of growing dotnet tools that helps to diagnose the application and do precise investigations. There is a great deal of them, but it is worth mentioning the most well-known dotnet tools:

• dotnet-counters — it is a global tool that monitors some counters (values) that should be measured, it elicits numeric values that could be used in performance analysis.
• dotnet-trace — it is a global tool that collects traces including events and CPU sampling, Garbage Collector collections, and database commands.
• dotnet-dump — it is a global tool that collects Windows/Linux dumps without involving any extra native debugger overheads.
• dotnet-gcdump — it is a global tool that collects Garbage Collector dump. It reveals objects’ roots and some general heap’s statistics.
• dotnet-monitor — it is a global tool that helps to get easy access to diagnostic information in dotnet processes. It units tools, such as dotnet-trace, dotnet-gcdump and dotnet-counters, and it also provides HTTP APIs for accessing those data. This approach simplifies the process of diagnosing an application wherever it has been running (Docker, K8s, local machine).

Recently, I started learning a new tool for benchmarking of multi-tiered applications (.NET and Docker)- Microsoft.Crank. I didn’t include it in the list above because I have not used it seriously on any project yet. Here is the link to a very good demo.

These dotnet tools may produce report files that can be later analyzed via a dotnet tool itself, in Visual Studio to some extent (not all analyzing features are available in VS), and finally, the most comprehensive tool for analyzing dumps, traces and samplings is PerfView (only Windows 😢).

All of these tools oblige profound .NET knowledge and some practice around that, it is especially relevant to low-level optimizations and performance improvements. Despite it, it could be beneficial to start using those tools in order to learn and understand .NET internals.

To sum up, I reckon that tools like NDepend significantly improve the process of designing complex .NET applications. Specifically, NDepend reduces the time spent on observing the application’s architecture, introduces new ways of checking (querying) the code, and gathers and shows important key indicators related to the code quality. Moreover, all checks can be done on a regular base by integrating NDepend into CICD pipelines. In the same way, the C4 modelling offloads our heads from keeping many details. It splits up the system architecture into several self-descriptive and concise diagrams that effectively convey the information to people with different backgrounds. And finally, the most sophisticated system’s parts require deliberate and precise engineering. And those listed tools are capable to do it very well. All these instruments do excellent jobs, but it could be even better if they would work together more coherently. In the nearest future, I am wondering to see more cross-platform tools that will extend each other’s capabilities through integrations. Maybe, we are going to see new standards for visualising, profiling and diagnosing that will facilitate different tools to work together and give an all-round experience.

Plug-in architecture is an attractive solution for developers seeking to build applications that are modular, customizable, and easily extensible. And a plug-in is a bundle that adds functionality to an application, called the host application, through some well-defined architecture for extensibility. This allows developers to add functionality to an application without having access to the source code.[1]

This article aims to show a simple approach how to extend a .Net application in a plug-in way.

Theoretical references

In this article, I decided to skip the theoretical part and jump into the implementation. However, my self-criticism didn’t give me a chance, and I provide some useful links below.

• There is a very brief and concise description of what plugin architecture is.
• Here is a very good material of this technique with good theoretical explanations of reasoning and use cases.

They are not too scientific, and I think they give just “enough” knowledge to go further.

Practice

There are many different ways how devs may approach this architecture, hence, some implementations are very strict or advanced, some of them are quite straightforward. I also have used several techniques to do that, I think the most common is to declare an interface and use it as a “gateway” between libraries. It is a versatile method, and everyone can use it as a swiss knife to build scenarios of any complexity(here is an example). However, I am going to show an even simpler option. The method is a bit naive and depends on the library, but it might be very handy at the right place and moment. Actually, when the MediatR library is in use. Instead of a developer, it does the dirty work, registers all handlers in given assemblies. So, some common commands and events should be defined in some shared projects. Then, plugin projects use these shared projects and implement the actual behavior. After that, these assemblies will be consumed by MediatR that will register all necessary handlers and their commands/events. The picture below depicts this process.

Registering handlers to corresponding commands and events

Actually, it bundles commands/events and handlers at runtime. That means that an application may re-bundle its components every run or every assembly load (it really depends on intention and implementation). Here is my implementation of registering things from loaded assemblies.

Registering handlers from given assemblies

As you can see, this approach is very simple and straightforward, especially, for people who are familiar with the MediatR library. I intentionally omitted all code activities in this article because they are very simple. This is a link where the source code of the given example might be found.

Conclusion

To sum up, this is a very simple and to some extent naive way of implementing the plug-in architecture. At the same time, it might be a very powerful and cheap technique if the project already uses the MediatR library. Of course, a decision of using it should be evaluated properly from many aspects, such as risks, project’s scope, maintainability, modifiability, etc. Then, based on pros, cons, and trade-offs the team may take it or just refuse.