Partly my goal was to create a guide that could be used by someone working in any field. But I was also curious about the shape of the intersection. And one thing this exercise shows is that it does have a definite shape; it's not just a point labelled "work hard."

The following recipe assumes you're very ambitious.

Choose what to work on

The first step is to decide what to work on. The work you choose needs to have three qualities: it has to be something you have a natural aptitude for, that you have a deep interest in, and that offers scope to do great work.

In practice you don't have to worry much about the third criterion. Ambitious people are if anything already too conservative about it. So all you need to do is find something you have an aptitude for and great interest in. [1]

That sounds straightforward, but it's often quite difficult. When you're young you don't know what you're good at or what different kinds of work are like. Some kinds of work you end up doing may not even exist yet. So while some people know what they want to do at 14, most have to figure it out.

The way to figure out what to work on is by working. If you're not sure what to work on, guess. But pick something and get going. You'll probably guess wrong some of the time, but that's fine. It's good to know about multiple things; some of the biggest discoveries come from noticing connections between different fields.

Develop a habit of working on your own projects. Don't let "work" mean something other people tell you to do. If you do manage to do great work one day, it will probably be on a project of your own. It may be within some bigger project, but you'll be driving your part of it.

What should your projects be? Whatever seems to you excitingly ambitious. As you grow older and your taste in projects evolves, exciting and important will converge. At 7 it may seem excitingly ambitious to build huge things out of Lego, then at 14 to teach yourself calculus, till at 21 you're starting to explore unanswered questions in physics. But always preserve excitingness.

There's a kind of excited curiosity that's both the engine and the rudder of great work. It will not only drive you, but if you let it have its way, will also show you what to work on.

What are you excessively curious about — curious to a degree that would bore most other people? That's what you're looking for.

Once you've found something you're excessively interested in, the next step is to learn enough about it to get you to one of the frontiers of knowledge. Knowledge expands fractally, and from a distance its edges look smooth, but once you learn enough to get close to one, they turn out to be full of gaps.

The next step is to notice them. This takes some skill, because your brain wants to ignore such gaps in order to make a simpler model of the world. Many discoveries have come from asking questions about things that everyone else took for granted. [2]

If the answers seem strange, so much the better. Great work often has a tincture of strangeness. You see this from painting to math. It would be affected to try to manufacture it, but if it appears, embrace it.

Boldly chase outlier ideas, even if other people aren't interested in them — in fact, especially if they aren't. If you're excited about some possibility that everyone else ignores, and you have enough expertise to say precisely what they're all overlooking, that's as good a bet as you'll find. [3]

Four steps: choose a field, learn enough to get to the frontier, notice gaps, explore promising ones. This is how practically everyone who's done great work has done it, from painters to physicists.

Steps two and four will require hard work. It may not be possible to prove that you have to work hard to do great things, but the empirical evidence is on the scale of the evidence for mortality. That's why it's essential to work on something you're deeply interested in. Interest will drive you to work harder than mere diligence ever could.

The three most powerful motives are curiosity, delight, and the desire to do something impressive. Sometimes they converge, and that combination is the most powerful of all.

The big prize is to discover a new fractal bud. You notice a crack in the surface of knowledge, pry it open, and there's a whole world inside.

Follow interestingness

Let's talk a little more about the complicated business of figuring out what to work on. The main reason it's hard is that you can't tell what most kinds of work are like except by doing them. Which means the four steps overlap: you may have to work at something for years before you know how much you like it or how good you are at it. And in the meantime you're not doing, and thus not learning about, most other kinds of work. So in the worst case you choose late based on very incomplete information. [4]

The nature of ambition exacerbates this problem. Ambition comes in two forms, one that precedes interest in the subject and one that grows out of it. Most people who do great work have a mix, and the more you have of the former, the harder it will be to decide what to do.

The educational systems in most countries pretend it's easy. They expect you to commit to a field long before you could know what it's really like. And as a result an ambitious person on an optimal trajectory will often read to the system as an instance of breakage.

It would be better if they at least admitted it — if they admitted that the system not only can't do much to help you figure out what to work on, but is designed on the assumption that you'll somehow magically guess as a teenager. They don't tell you, but I will: when it comes to figuring out what to work on, you're on your own. Some people get lucky and do guess correctly, but the rest will find themselves scrambling diagonally across tracks laid down on the assumption that everyone does.

What should you do if you're young and ambitious but don't know what to work on? What you should not do is drift along passively, assuming the problem will solve itself. You need to take action. But there is no systematic procedure you can follow. When you read biographies of people who've done great work, it's remarkable how much luck is involved. They discover what to work on as a result of a chance meeting, or by reading a book they happen to pick up. So you need to make yourself a big target for luck, and the way to do that is to be curious. Try lots of things, meet lots of people, read lots of books, ask lots of questions. [5]

When in doubt, optimize for interestingness. Fields change as you learn more about them. What mathematicians do, for example, is very different from what you do in high school math classes. So you need to give different types of work a chance to show you what they're like. But a field should become increasingly interesting as you learn more about it. If it doesn't, it's probably not for you.

Don't worry if you find you're interested in different things than other people. The stranger your tastes in interestingness, the better. Strange tastes are often strong ones, and a strong taste for work means you'll be productive. And you're more likely to find new things if you're looking where few have looked before.

One sign that you're suited for some kind of work is when you like even the parts that other people find tedious or frightening.

But fields aren't people; you don't owe them any loyalty. If in the course of working on one thing you discover another that's more exciting, don't be afraid to switch.

If you're making something for people, make sure it's something they actually want. The best way to do this is to make something you yourself want. Write the story you want to read; build the tool you want to use. Since your friends probably have similar interests, this will also get you your initial audience.

This should follow from the excitingness rule. Obviously the most exciting story to write will be the one you want to read. The reason I mention this case explicitly is that so many people get it wrong. Instead of making what they want, they try to make what some imaginary, more sophisticated audience wants. And once you go down that route, you're lost. [6]

There are a lot of forces that will lead you astray when you're trying to figure out what to work on. Pretentiousness, fashion, fear, money, politics, other people's wishes, eminent frauds. But if you stick to what you find genuinely interesting, you'll be proof against all of them. If you're interested, you're not astray.

Following your interests may sound like a rather passive strategy, but in practice it usually means following them past all sorts of obstacles. You usually have to risk rejection and failure. So it does take a good deal of boldness.

But while you need boldness, you don't usually need much planning. In most cases the recipe for doing great work is simply: work hard on excitingly ambitious projects, and something good will come of it. Instead of making a plan and then executing it, you just try to preserve certain invariants.

The trouble with planning is that it only works for achievements you can describe in advance. You can win a gold medal or get rich by deciding to as a child and then tenaciously pursuing that goal, but you can't discover natural selection that way.

I think for most people who want to do great work, the right strategy is not to plan too much. At each stage do whatever seems most interesting and gives you the best options for the future. I call this approach "staying upwind." This is how most people who've done great work seem to have done it.

Work hard without burning out

Even when you've found something exciting to work on, working on it is not always straightforward. There will be times when some new idea makes you leap out of bed in the morning and get straight to work. But there will also be plenty of times when things aren't like that.

You don't just put out your sail and get blown forward by inspiration. There are headwinds and currents and hidden shoals. So there's a technique to working, just as there is to sailing.

For example, while you must work hard, it's possible to work too hard, and if you do that you'll find you get diminishing returns: fatigue will make you stupid, and eventually even damage your health. The point at which work yields diminishing returns depends on the type. Some of the hardest types you might only be able to do for four or five hours a day.

Ideally those hours will be contiguous. To the extent you can, try to arrange your life so you have big blocks of time to work in. You'll shy away from hard tasks if you know you might be interrupted.

It will probably be harder to start working than to keep working. You'll often have to trick yourself to get over that initial threshold. Don't worry about this; it's the nature of work, not a flaw in your character. Work has a sort of activation energy, both per day and per project. And since this threshold is fake in the sense that it's higher than the energy required to keep going, it's ok to tell yourself a lie of corresponding magnitude to get over it.

It's usually a mistake to lie to yourself if you want to do great work, but this is one of the rare cases where it isn't. When I'm reluctant to start work in the morning, I often trick myself by saying "I'll just read over what I've got so far." Five minutes later I've found something that seems mistaken or incomplete, and I'm off.

Similar techniques work for starting new projects. It's ok to lie to yourself about how much work a project will entail, for example. Lots of great things began with someone saying "How hard could it be?"

This is one case where the young have an advantage. They're more optimistic, and even though one of the sources of their optimism is ignorance, in this case ignorance can sometimes beat knowledge.

Try to finish what you start, though, even if it turns out to be more work than you expected. Finishing things is not just an exercise in tidiness or self-discipline. In many projects a lot of the best work happens in what was meant to be the final stage.

Another permissible lie is to exaggerate the importance of what you're working on, at least in your own mind. If that helps you discover something new, it may turn out not to have been a lie at all. [7]

Since there are two senses of starting work — per day and per project — there are also two forms of procrastination. Per-project procrastination is far the more dangerous. You put off starting that ambitious project from year to year because the time isn't quite right. When you're procrastinating in units of years, you can get a lot not done. [8]

One reason per-project procrastination is so dangerous is that it usually camouflages itself as work. You're not just sitting around doing nothing; you're working industriously on something else. So per-project procrastination doesn't set off the alarms that per-day procrastination does. You're too busy to notice it.

The way to beat it is to stop occasionally and ask yourself: Am I working on what I most want to work on? When you're young it's ok if the answer is sometimes no, but this gets increasingly dangerous as you get older. [9]

Great work usually entails spending what would seem to most people an unreasonable amount of time on a problem. You can't think of this time as a cost, or it will seem too high. You have to find the work sufficiently engaging as it's happening.

There may be some jobs where you have to work diligently for years at things you hate before you get to the good part, but this is not how great work happens. Great work happens by focusing consistently on something you're genuinely interested in. When you pause to take stock, you're surprised how far you've come.

The reason we're surprised is that we underestimate the cumulative effect of work. Writing a page a day doesn't sound like much, but if you do it every day you'll write a book a year. That's the key: consistency. People who do great things don't get a lot done every day. They get something done, rather than nothing.

If you do work that compounds, you'll get exponential growth. Most people who do this do it unconsciously, but it's worth stopping to think about. Learning, for example, is an instance of this phenomenon: the more you learn about something, the easier it is to learn more. Growing an audience is another: the more fans you have, the more new fans they'll bring you.

The trouble with exponential growth is that the curve feels flat in the beginning. It isn't; it's still a wonderful exponential curve. But we can't grasp that intuitively, so we underrate exponential growth in its early stages.

Something that grows exponentially can become so valuable that it's worth making an extraordinary effort to get it started. But since we underrate exponential growth early on, this too is mostly done unconsciously: people push through the initial, unrewarding phase of learning something new because they know from experience that learning new things always takes an initial push, or they grow their audience one fan at a time because they have nothing better to do. If people consciously realized they could invest in exponential growth, many more would do it.

Work doesn't just happen when you're trying to. There's a kind of undirected thinking you do when walking or taking a shower or lying in bed that can be very powerful. By letting your mind wander a little, you'll often solve problems you were unable to solve by frontal attack.

You have to be working hard in the normal way to benefit from this phenomenon, though. You can't just walk around daydreaming. The daydreaming has to be interleaved with deliberate work that feeds it questions. [10]

Everyone knows to avoid distractions at work, but it's also important to avoid them in the other half of the cycle. When you let your mind wander, it wanders to whatever you care about most at that moment. So avoid the kind of distraction that pushes your work out of the top spot, or you'll waste this valuable type of thinking on the distraction instead. (Exception: Don't avoid love.)

Cultivate taste and earnestness

Consciously cultivate your taste in the work done in your field. Until you know which is the best and what makes it so, you don't know what you're aiming for.

And that is what you're aiming for, because if you don't try to be the best, you won't even be good. This observation has been made by so many people in so many different fields that it might be worth thinking about why it's true. It could be because ambition is a phenomenon where almost all the error is in one direction — where almost all the shells that miss the target miss by falling short. Or it could be because ambition to be the best is a qualitatively different thing from ambition to be good. Or maybe being good is simply too vague a standard. Probably all three are true. [11]

Fortunately there's a kind of economy of scale here. Though it might seem like you'd be taking on a heavy burden by trying to be the best, in practice you often end up net ahead. It's exciting, and also strangely liberating. It simplifies things. In some ways it's easier to try to be the best than to try merely to be good.

One way to aim high is to try to make something that people will care about in a hundred years. Not because their opinions matter more than your contemporaries', but because something that still seems good in a hundred years is more likely to be genuinely good.

Don't try to work in a distinctive style. Just try to do the best job you can; you won't be able to help doing it in a distinctive way.

Style is doing things in a distinctive way without trying to. Trying to is affectation.

Affectation is in effect to pretend that someone other than you is doing the work. You adopt an impressive but fake persona, and while you're pleased with the impressiveness, the fakeness is what shows in the work. [12]

The temptation to be someone else is greatest for the young. They often feel like nobodies. But you never need to worry about that problem, because it's self-solving if you work on sufficiently ambitious projects. If you succeed at an ambitious project, you're not a nobody; you're the person who did it. So just do the work and your identity will take care of itself.

"Avoid affectation" is a useful rule so far as it goes, but how would you express this idea positively? How would you say what to be, instead of what not to be? The best answer is earnest. If you're earnest you avoid not just affectation but a whole set of similar vices.

The core of being earnest is being intellectually honest. We're taught as children to be honest as an unselfish virtue — as a kind of sacrifice. But in fact it's a source of power too. To see new ideas, you need an exceptionally sharp eye for the truth. You're trying to see more truth than others have seen so far. And how can you have a sharp eye for the truth if you're intellectually dishonest?

One way to avoid intellectual dishonesty is to maintain a slight positive pressure in the opposite direction. Be aggressively willing to admit that you're mistaken. Once you've admitted you were mistaken about something, you're free. Till then you have to carry it. [13]

Another more subtle component of earnestness is informality. Informality is much more important than its grammatically negative name implies. It's not merely the absence of something. It means focusing on what matters instead of what doesn't.

What formality and affectation have in common is that as well as doing the work, you're trying to seem a certain way as you're doing it. But any energy that goes into how you seem comes out of being good. That's one reason nerds have an advantage in doing great work: they expend little effort on seeming anything. In fact that's basically the definition of a nerd.

Nerds have a kind of innocent boldness that's exactly what you need in doing great work. It's not learned; it's preserved from childhood. So hold onto it. Be the one who puts things out there rather than the one who sits back and offers sophisticated-sounding criticisms of them. "It's easy to criticize" is true in the most literal sense, and the route to great work is never easy.

There may be some jobs where it's an advantage to be cynical and pessimistic, but if you want to do great work it's an advantage to be optimistic, even though that means you'll risk looking like a fool sometimes. There's an old tradition of doing the opposite. The Old Testament says it's better to keep quiet lest you look like a fool. But that's advice for seeming smart. If you actually want to discover new things, it's better to take the risk of telling people your ideas.

Some people are naturally earnest, and with others it takes a conscious effort. Either kind of earnestness will suffice. But I doubt it would be possible to do great work without being earnest. It's so hard to do even if you are. You don't have enough margin for error to accommodate the distortions introduced by being affected, intellectually dishonest, orthodox, fashionable, or cool. [14]

Great work is consistent not only with who did it, but with itself. It's usually all of a piece. So if you face a decision in the middle of working on something, ask which choice is more consistent.

You may have to throw things away and redo them. You won't necessarily have to, but you have to be willing to. And that can take some effort; when there's something you need to redo, status quo bias and laziness will combine to keep you in denial about it. To beat this ask: If I'd already made the change, would I want to revert to what I have now?

Have the confidence to cut. Don't keep something that doesn't fit just because you're proud of it, or because it cost you a lot of effort.

Indeed, in some kinds of work it's good to strip whatever you're doing to its essence. The result will be more concentrated; you'll understand it better; and you won't be able to lie to yourself about whether there's anything real there.

Mathematical elegance may sound like a mere metaphor, drawn from the arts. That's what I thought when I first heard the term "elegant" applied to a proof. But now I suspect it's conceptually prior — that the main ingredient in artistic elegance is mathematical elegance. At any rate it's a useful standard well beyond math.

Elegance can be a long-term bet, though. Laborious solutions will often have more prestige in the short term. They cost a lot of effort and they're hard to understand, both of which impress people, at least temporarily.

Whereas some of the very best work will seem like it took comparatively little effort, because it was in a sense already there. It didn't have to be built, just seen. It's a very good sign when it's hard to say whether you're creating something or discovering it.

When you're doing work that could be seen as either creation or discovery, err on the side of discovery. Try thinking of yourself as a mere conduit through which the ideas take their natural shape.

(Strangely enough, one exception is the problem of choosing a problem to work on. This is usually seen as search, but in the best case it's more like creating something. In the best case you create the field in the process of exploring it.)

Similarly, if you're trying to build a powerful tool, make it gratuitously unrestrictive. A powerful tool almost by definition will be used in ways you didn't expect, so err on the side of eliminating restrictions, even if you don't know what the benefit will be.

Great work will often be tool-like in the sense of being something others build on. So it's a good sign if you're creating ideas that others could use, or exposing questions that others could answer. The best ideas have implications in many different areas.

If you express your ideas in the most general form, they'll be truer than you intended.

Originality and broken models

True by itself is not enough, of course. Great ideas have to be true and new. And it takes a certain amount of ability to see new ideas even once you've learned enough to get to one of the frontiers of knowledge.

In English we give this ability names like originality, creativity, and imagination. And it seems reasonable to give it a separate name, because it does seem to some extent a separate skill. It's possible to have a great deal of ability in other respects — to have a great deal of what's often called technical ability — and yet not have much of this.

I've never liked the term "creative process." It seems misleading. Originality isn't a process, but a habit of mind. Original thinkers throw off new ideas about whatever they focus on, like an angle grinder throwing off sparks. They can't help it.

If the thing they're focused on is something they don't understand very well, these new ideas might not be good. One of the most original thinkers I know decided to focus on dating after he got divorced. He knew roughly as much about dating as the average 15 year old, and the results were spectacularly colorful. But to see originality separated from expertise like that made its nature all the more clear.

I don't know if it's possible to cultivate originality, but there are definitely ways to make the most of however much you have. For example, you're much more likely to have original ideas when you're working on something. Original ideas don't come from trying to have original ideas. They come from trying to build or understand something slightly too difficult. [15]

Talking or writing about the things you're interested in is a good way to generate new ideas. When you try to put ideas into words, a missing idea creates a sort of vacuum that draws it out of you. Indeed, there's a kind of thinking that can only be done by writing.

Changing your context can help. If you visit a new place, you'll often find you have new ideas there. The journey itself often dislodges them. But you may not have to go far to get this benefit. Sometimes it's enough just to go for a walk. [16]

It also helps to travel in topic space. You'll have more new ideas if you explore lots of different topics, partly because it gives the angle grinder more surface area to work on, and partly because analogies are an especially fruitful source of new ideas.

Don't divide your attention evenly between many topics though, or you'll spread yourself too thin. You want to distribute it according to something more like a power law. [17] Be professionally curious about a few topics and idly curious about many more.

Curiosity and originality are closely related. Curiosity feeds originality by giving it new things to work on. But the relationship is closer than that. Curiosity is itself a kind of originality; it's roughly to questions what originality is to answers. And since questions at their best are a big component of answers, curiosity at its best is a creative force.

Having new ideas is a strange game, because it usually consists of seeing things that were right under your nose. Once you've seen a new idea, it tends to seem obvious. Why did no one think of this before?

When an idea seems simultaneously novel and obvious, it's probably a good one.

Seeing something obvious sounds easy. And yet empirically having new ideas is hard. What's the source of this apparent contradiction? It's that seeing the new idea usually requires you to change the way you look at the world. We see the world through models that both help and constrain us. When you fix a broken model, new ideas become obvious. But noticing and fixing a broken model is hard. That's how new ideas can be both obvious and yet hard to discover: they're easy to see after you do something hard.

One way to discover broken models is to be stricter than other people. Broken models of the world leave a trail of clues where they bash against reality. Most people don't want to see these clues. It would be an understatement to say that they're attached to their current model; it's what they think in; so they'll tend to ignore the trail of clues left by its breakage, however conspicuous it may seem in retrospect.

To find new ideas you have to seize on signs of breakage instead of looking away. That's what Einstein did. He was able to see the wild implications of Maxwell's equations not so much because he was looking for new ideas as because he was stricter.

The other thing you need is a willingness to break rules. Paradoxical as it sounds, if you want to fix your model of the world, it helps to be the sort of person who's comfortable breaking rules. From the point of view of the old model, which everyone including you initially shares, the new model usually breaks at least implicit rules.

Few understand the degree of rule-breaking required, because new ideas seem much more conservative once they succeed. They seem perfectly reasonable once you're using the new model of the world they brought with them. But they didn't at the time; it took the greater part of a century for the heliocentric model to be generally accepted, even among astronomers, because it felt so wrong.

Indeed, if you think about it, a good new idea has to seem bad to most people, or someone would have already explored it. So what you're looking for is ideas that seem crazy, but the right kind of crazy. How do you recognize these? You can't with certainty. Often ideas that seem bad are bad. But ideas that are the right kind of crazy tend to be exciting; they're rich in implications; whereas ideas that are merely bad tend to be depressing.

There are two ways to be comfortable breaking rules: to enjoy breaking them, and to be indifferent to them. I call these two cases being aggressively and passively independent-minded.

The aggressively independent-minded are the naughty ones. Rules don't merely fail to stop them; breaking rules gives them additional energy. For this sort of person, delight at the sheer audacity of a project sometimes supplies enough activation energy to get it started.

The other way to break rules is not to care about them, or perhaps even to know they exist. This is why novices and outsiders often make new discoveries; their ignorance of a field's assumptions acts as a source of temporary passive independent-mindedness. Aspies also seem to have a kind of immunity to conventional beliefs. Several I know say that this helps them to have new ideas.

Strictness plus rule-breaking sounds like a strange combination. In popular culture they're opposed. But popular culture has a broken model in this respect. It implicitly assumes that issues are trivial ones, and in trivial matters strictness and rule-breaking are opposed. But in questions that really matter, only rule-breakers can be truly strict.

An overlooked idea often doesn't lose till the semifinals. You do see it, subconsciously, but then another part of your subconscious shoots it down because it would be too weird, too risky, too much work, too controversial. This suggests an exciting possibility: if you could turn off such filters, you could see more new ideas.

One way to do that is to ask what would be good ideas for someone else to explore. Then your subconscious won't shoot them down to protect you.

You could also discover overlooked ideas by working in the other direction: by starting from what's obscuring them. Every cherished but mistaken principle is surrounded by a dead zone of valuable ideas that are unexplored because they contradict it.

Religions are collections of cherished but mistaken principles. So anything that can be described either literally or metaphorically as a religion will have valuable unexplored ideas in its shadow. Copernicus and Darwin both made discoveries of this type. [18]

What are people in your field religious about, in the sense of being too attached to some principle that might not be as self-evident as they think? What becomes possible if you discard it?

People show much more originality in solving problems than in deciding which problems to solve. Even the smartest can be surprisingly conservative when deciding what to work on. People who'd never dream of being fashionable in any other way get sucked into working on fashionable problems.

One reason people are more conservative when choosing problems than solutions is that problems are bigger bets. A problem could occupy you for years, while exploring a solution might only take days. But even so I think most people are too conservative. They're not merely responding to risk, but to fashion as well. Unfashionable problems are undervalued.

One of the most interesting kinds of unfashionable problem is the problem that people think has been fully explored, but hasn't. Great work often takes something that already exists and shows its latent potential. Durer and Watt both did this. So if you're interested in a field that others think is tapped out, don't let their skepticism deter you. People are often wrong about this.

Working on an unfashionable problem can be very pleasing. There's no hype or hurry. Opportunists and critics are both occupied elsewhere. The existing work often has an old-school solidity. And there's a satisfying sense of economy in cultivating ideas that would otherwise be wasted.

But the most common type of overlooked problem is not explicitly unfashionable in the sense of being out of fashion. It just doesn't seem to matter as much as it actually does. How do you find these? By being self-indulgent — by letting your curiosity have its way, and tuning out, at least temporarily, the little voice in your head that says you should only be working on "important" problems.

You do need to work on important problems, but almost everyone is too conservative about what counts as one. And if there's an important but overlooked problem in your neighborhood, it's probably already on your subconscious radar screen. So try asking yourself: if you were going to take a break from "serious" work to work on something just because it would be really interesting, what would you do? The answer is probably more important than it seems.

Originality in choosing problems seems to matter even more than originality in solving them. That's what distinguishes the people who discover whole new fields. So what might seem to be merely the initial step — deciding what to work on — is in a sense the key to the whole game.

Few grasp this. One of the biggest misconceptions about new ideas is about the ratio of question to answer in their composition. People think big ideas are answers, but often the real insight was in the question.

Part of the reason we underrate questions is the way they're used in schools. In schools they tend to exist only briefly before being answered, like unstable particles. But a really good question can be much more than that. A really good question is a partial discovery. How do new species arise? Is the force that makes objects fall to earth the same as the one that keeps planets in their orbits? By even asking such questions you were already in excitingly novel territory.

Unanswered questions can be uncomfortable things to carry around with you. But the more you're carrying, the greater the chance of noticing a solution — or perhaps even more excitingly, noticing that two unanswered questions are the same.

Sometimes you carry a question for a long time. Great work often comes from returning to a question you first noticed years before — in your childhood, even — and couldn't stop thinking about. People talk a lot about the importance of keeping your youthful dreams alive, but it's just as important to keep your youthful questions alive. [19]

This is one of the places where actual expertise differs most from the popular picture of it. In the popular picture, experts are certain. But actually the more puzzled you are, the better, so long as (a) the things you're puzzled about matter, and (b) no one else understands them either.

Think about what's happening at the moment just before a new idea is discovered. Often someone with sufficient expertise is puzzled about something. Which means that originality consists partly of puzzlement — of confusion! You have to be comfortable enough with the world being full of puzzles that you're willing to see them, but not so comfortable that you don't want to solve them. [20]

It's a great thing to be rich in unanswered questions. And this is one of those situations where the rich get richer, because the best way to acquire new questions is to try answering existing ones. Questions don't just lead to answers, but also to more questions.

The best questions grow in the answering. You notice a thread protruding from the current paradigm and try pulling on it, and it just gets longer and longer. So don't require a question to be obviously big before you try answering it. You can rarely predict that. It's hard enough even to notice the thread, let alone to predict how much will unravel if you pull on it.

Start small and iterate

It's better to be promiscuously curious — to pull a little bit on a lot of threads, and see what happens. Big things start small. The initial versions of big things were often just experiments, or side projects, or talks, which then grew into something bigger. So start lots of small things.

Being prolific is underrated. The more different things you try, the greater the chance of discovering something new. Understand, though, that trying lots of things will mean trying lots of things that don't work. You can't have a lot of good ideas without also having a lot of bad ones. [21]

Though it sounds more responsible to begin by studying everything that's been done before, you'll learn faster and have more fun by trying stuff. And you'll understand previous work better when you do look at it. So err on the side of starting. Which is easier when starting means starting small; those two ideas fit together like two puzzle pieces.

How do you get from starting small to doing something great? By making successive versions. Great things are almost always made in successive versions. You start with something small and evolve it, and the final version is both cleverer and more ambitious than anything you could have planned.

It's particularly useful to make successive versions when you're making something for people — to get an initial version in front of them quickly, and then evolve it based on their response.

Begin by trying the simplest thing that could possibly work. Surprisingly often, it does. If it doesn't, this will at least get you started.

Don't try to cram too much new stuff into any one version. There are names for doing this with the first version (taking too long to ship) and the second (the second system effect), but these are both merely instances of a more general principle.

An early version of a new project will sometimes be dismissed as a toy. It's a good sign when people do this. That means it has everything a new idea needs except scale, and that tends to follow. [22]

The alternative to starting with something small and evolving it is to plan in advance what you're going to do. And planning does usually seem the more responsible choice. It sounds more organized to say "we're going to do x and then y and then z" than "we're going to try x and see what happens." And it is more organized; it just doesn't work as well.

Planning per se isn't good. It's sometimes necessary, but it's a necessary evil — a response to unforgiving conditions. It's something you have to do because you're working with inflexible media, or because you need to coordinate the efforts of a lot of people. If you keep projects small and use flexible media, you don't have to plan as much, and your designs can evolve instead.

Take as much risk as you can afford. In an efficient market, risk is proportionate to reward, so don't look for certainty, but for a bet with high expected value. If you're not failing occasionally, you're probably being too conservative.

Though conservatism is usually associated with the old, it's the young who tend to make this mistake. Inexperience makes them fear risk, but it's when you're young that you can afford the most.

Even a project that fails can be valuable. In the process of working on it, you'll have crossed territory few others have seen, and encountered questions few others have asked. And there's probably no better source of questions than the ones you encounter in trying to do something slightly too hard.

Learn with fresh eyes

Use the advantages of youth when you have them, and the advantages of age once you have those. The advantages of youth are energy, time, optimism, and freedom. The advantages of age are knowledge, efficiency, money, and power. With effort you can acquire some of the latter when young and keep some of the former when old.

The old also have the advantage of knowing which advantages they have. The young often have them without realizing it. The biggest is probably time. The young have no idea how rich they are in time. The best way to turn this time to advantage is to use it in slightly frivolous ways: to learn about something you don't need to know about, just out of curiosity, or to try building something just because it would be cool, or to become freakishly good at something.

That "slightly" is an important qualification. Spend time lavishly when you're young, but don't simply waste it. There's a big difference between doing something you worry might be a waste of time and doing something you know for sure will be. The former is at least a bet, and possibly a better one than you think. [23]

The most subtle advantage of youth, or more precisely of inexperience, is that you're seeing everything with fresh eyes. When your brain embraces an idea for the first time, sometimes the two don't fit together perfectly. Usually the problem is with your brain, but occasionally it's with the idea. A piece of it sticks out awkwardly and jabs you when you think about it. People who are used to the idea have learned to ignore it, but you have the opportunity not to. [24]

So when you're learning about something for the first time, pay attention to things that seem wrong or missing. You'll be tempted to ignore them, since there's a 99% chance the problem is with you. And you may have to set aside your misgivings temporarily to keep progressing. But don't forget about them. When you've gotten further into the subject, come back and check if they're still there. If they're still viable in the light of your present knowledge, they probably represent an undiscovered idea.

One of the most valuable kinds of knowledge you get from experience is to know what you don't have to worry about. The young know all the things that could matter, but not their relative importance. So they worry equally about everything, when they should worry much more about a few things and hardly at all about the rest.

But what you don't know is only half the problem with inexperience. The other half is what you do know that ain't so. You arrive at adulthood with your head full of nonsense — bad habits you've acquired and false things you've been taught — and you won't be able to do great work till you clear away at least the nonsense in the way of whatever type of work you want to do.

Much of the nonsense left in your head is left there by schools. We're so used to schools that we unconsciously treat going to school as identical with learning, but in fact schools have all sorts of strange qualities that warp our ideas about learning and thinking.

For example, schools induce passivity. Since you were a small child, there was an authority at the front of the class telling all of you what you had to learn and then measuring whether you did. But neither classes nor tests are intrinsic to learning; they're just artifacts of the way schools are usually designed.

The sooner you overcome this passivity, the better. If you're still in school, try thinking of your education as your project, and your teachers as working for you rather than vice versa. That may seem a stretch, but it's not merely some weird thought experiment. It's the truth economically, and in the best case it's the truth intellectually as well. The best teachers don't want to be your bosses. They'd prefer it if you pushed ahead, using them as a source of advice, rather than being pulled by them through the material.

Schools also give you a misleading impression of what work is like. In school they tell you what the problems are, and they're almost always soluble using no more than you've been taught so far. In real life you have to figure out what the problems are, and you often don't know if they're soluble at all.

But perhaps the worst thing schools do to you is train you to win by hacking the test. You can't do great work by doing that. You can't trick God. So stop looking for that kind of shortcut. The way to beat the system is to focus on problems and solutions that others have overlooked, not to skimp on the work itself.

Don't think of yourself as dependent on some gatekeeper giving you a "big break." Even if this were true, the best way to get it would be to focus on doing good work rather than chasing influential people.

And don't take rejection by committees to heart. The qualities that impress admissions officers and prize committees are quite different from those required to do great work. The decisions of selection committees are only meaningful to the extent that they're part of a feedback loop, and very few are.

People new to a field will often copy existing work. There's nothing inherently bad about that. There's no better way to learn how something works than by trying to reproduce it. Nor does copying necessarily make your work unoriginal. Originality is the presence of new ideas, not the absence of old ones.

There's a good way to copy and a bad way. If you're going to copy something, do it openly instead of furtively, or worse still, unconsciously. This is what's meant by the famously misattributed phrase "Great artists steal." The really dangerous kind of copying, the kind that gives copying a bad name, is the kind that's done without realizing it, because you're nothing more than a train running on tracks laid down by someone else. But at the other extreme, copying can be a sign of superiority rather than subordination. [25]

In many fields it's almost inevitable that your early work will be in some sense based on other people's. Projects rarely arise in a vacuum. They're usually a reaction to previous work. When you're first starting out, you don't have any previous work; if you're going to react to something, it has to be someone else's. Once you're established, you can react to your own. But while the former gets called derivative and the latter doesn't, structurally the two cases are more similar than they seem.

Oddly enough, the very novelty of the most novel ideas sometimes makes them seem at first to be more derivative than they are. New discoveries often have to be conceived initially as variations of existing things, even by their discoverers, because there isn't yet the conceptual vocabulary to express them.

There are definitely some dangers to copying, though. One is that you'll tend to copy old things — things that were in their day at the frontier of knowledge, but no longer are.

And when you do copy something, don't copy every feature of it. Some will make you ridiculous if you do. Don't copy the manner of an eminent 50 year old professor if you're 18, for example, or the idiom of a Renaissance poem hundreds of years later.

Some of the features of things you admire are flaws they succeeded despite. Indeed, the features that are easiest to imitate are the most likely to be the flaws.

This is particularly true for behavior. Some talented people are jerks, and this sometimes makes it seem to the inexperienced that being a jerk is part of being talented. It isn't; being talented is merely how they get away with it.

One of the most powerful kinds of copying is to copy something from one field into another. History is so full of chance discoveries of this type that it's probably worth giving chance a hand by deliberately learning about other kinds of work. You can take ideas from quite distant fields if you let them be metaphors.

Negative examples can be as inspiring as positive ones. In fact you can sometimes learn more from things done badly than from things done well; sometimes it only becomes clear what's needed when it's missing.

If a lot of the best people in your field are collected in one place, it's usually a good idea to visit for a while. It will increase your ambition, and also, by showing you that these people are human, increase your self-confidence. [26]

If you're earnest you'll probably get a warmer welcome than you might expect. Most people who are very good at something are happy to talk about it with anyone who's genuinely interested. If they're really good at their work, then they probably have a hobbyist's interest in it, and hobbyists always want to talk about their hobbies.

It may take some effort to find the people who are really good, though. Doing great work has such prestige that in some places, particularly universities, there's a polite fiction that everyone is engaged in it. And that is far from true. People within universities can't say so openly, but the quality of the work being done in different departments varies immensely. Some departments have people doing great work; others have in the past; others never have.

Seek out the best colleagues. There are a lot of projects that can't be done alone, and even if you're working on one that can, it's good to have other people to encourage you and to bounce ideas off.

Colleagues don't just affect your work, though; they also affect you. So work with people you want to become like, because you will.

Quality is more important than quantity in colleagues. It's better to have one or two great ones than a building full of pretty good ones. In fact it's not merely better, but necessary, judging from history: the degree to which great work happens in clusters suggests that one's colleagues often make the difference between doing great work and not.

How do you know when you have sufficiently good colleagues? In my experience, when you do, you know. Which means if you're unsure, you probably don't. But it may be possible to give a more concrete answer than that. Here's an attempt: sufficiently good colleagues offer surprising insights. They can see and do things that you can't. So if you have a handful of colleagues good enough to keep you on your toes in this sense, you're probably over the threshold.

Most of us can benefit from collaborating with colleagues, but some projects require people on a larger scale, and starting one of those is not for everyone. If you want to run a project like that, you'll have to become a manager, and managing well takes aptitude and interest like any other kind of work. If you don't have them, there is no middle path: you must either force yourself to learn management as a second language, or avoid such projects. [27]

Protect your morale

Husband your morale. It's the basis of everything when you're working on ambitious projects. You have to nurture and protect it like a living organism.

Morale starts with your view of life. You're more likely to do great work if you're an optimist, and more likely to if you think of yourself as lucky than if you think of yourself as a victim.

Indeed, work can to some extent protect you from your problems. If you choose work that's pure, its very difficulties will serve as a refuge from the difficulties of everyday life. If this is escapism, it's a very productive form of it, and one that has been used by some of the greatest minds in history.

Morale compounds via work: high morale helps you do good work, which increases your morale and helps you do even better work. But this cycle also operates in the other direction: if you're not doing good work, that can demoralize you and make it even harder to. Since it matters so much for this cycle to be running in the right direction, it can be a good idea to switch to easier work when you're stuck, just so you start to get something done.

One of the biggest mistakes ambitious people make is to allow setbacks to destroy their morale all at once, like a balloon bursting. You can inoculate yourself against this by explicitly considering setbacks a part of your process. Solving hard problems always involves some backtracking.

Doing great work is a depth-first search whose root node is the desire to. So "If at first you don't succeed, try, try again" isn't quite right. It should be: If at first you don't succeed, either try again, or backtrack and then try again.

"Never give up" is also not quite right. Obviously there are times when it's the right choice to eject. A more precise version would be: Never let setbacks panic you into backtracking more than you need to. Corollary: Never abandon the root node.

It's not necessarily a bad sign if work is a struggle, any more than it's a bad sign to be out of breath while running. It depends how fast you're running. So learn to distinguish good pain from bad. Good pain is a sign of effort; bad pain is a sign of damage.

An audience is a critical component of morale. If you're a scholar, your audience may be your peers; in the arts, it may be an audience in the traditional sense. Either way it doesn't need to be big. The value of an audience doesn't grow anything like linearly with its size. Which is bad news if you're famous, but good news if you're just starting out, because it means a small but dedicated audience can be enough to sustain you. If a handful of people genuinely love what you're doing, that's enough.

To the extent you can, avoid letting intermediaries come between you and your audience. In some types of work this is inevitable, but it is so liberating to escape it that you might be better off switching to an adjacent type if that will let you go direct. [28]

The people you spend time with will also have a big effect on your morale. You'll find there are some who increase your energy and others who decrease it, and the effect someone has is not always what you'd expect. Seek out the people who increase your energy and avoid those who decrease it. Though of course if there's someone you need to take care of, that takes precedence.

Don't marry someone who doesn't understand that you need to work, or sees your work as competition for your attention. If you're ambitious, you need to work; it's almost like a medical condition; so someone who won't let you work either doesn't understand you, or does and doesn't care.

Ultimately morale is physical. You think with your body, so it's important to take care of it. That means exercising regularly, eating and sleeping well, and avoiding the more dangerous kinds of drugs. Running and walking are particularly good forms of exercise because they're good for thinking. [29]

People who do great work are not necessarily happier than everyone else, but they're happier than they'd be if they didn't. In fact, if you're smart and ambitious, it's dangerous not to be productive. People who are smart and ambitious but don't achieve much tend to become bitter.

It's ok to want to impress other people, but choose the right people. The opinion of people you respect is signal. Fame, which is the opinion of a much larger group you might or might not respect, just adds noise.

The prestige of a type of work is at best a trailing indicator and sometimes completely mistaken. If you do anything well enough, you'll make it prestigious. So the question to ask about a type of work is not how much prestige it has, but how well it could be done.

Competition can be an effective motivator, but don't let it choose the problem for you; don't let yourself get drawn into chasing something just because others are. In fact, don't let competitors make you do anything much more specific than work harder.

Let curiosity lead

Curiosity is the best guide. Your curiosity never lies, and it knows more than you do about what's worth paying attention to.

Notice how often that word has come up. If you asked an oracle the secret to doing great work and the oracle replied with a single word, my bet would be on "curiosity."

That doesn't translate directly to advice. It's not enough just to be curious, and you can't command curiosity anyway. But you can nurture it and let it drive you.

Curiosity is the key to all four steps in doing great work: it will choose the field for you, get you to the frontier, cause you to notice the gaps in it, and drive you to explore them. The whole process is a kind of dance with curiosity.

Believe it or not, I tried to make this essay as short as I could. But its length at least means it acts as a filter. If you made it this far, you must be interested in doing great work. And if so you're already further along than you might realize, because the set of people willing to want to is small.

The factors in doing great work are factors in the literal, mathematical sense, and they are: ability, interest, effort, and luck. Luck by definition you can't do anything about, so we can ignore that. And we can assume effort, if you do in fact want to do great work. So the problem boils down to ability and interest. Can you find a kind of work where your ability and interest will combine to yield an explosion of new ideas?

Here there are grounds for optimism. There are so many different ways to do great work, and even more that are still undiscovered. Out of all those different types of work, the one you're most suited for is probably a pretty close match. Probably a comically close match. It's just a question of finding it, and how far into it your ability and interest can take you. And you can only answer that by trying.

Many more people could try to do great work than do. What holds them back is a combination of modesty and fear. It seems presumptuous to try to be Newton or Shakespeare. It also seems hard; surely if you tried something like that, you'd fail. Presumably the calculation is rarely explicit. Few people consciously decide not to try to do great work. But that's what's going on subconsciously; they shy away from the question.

So I'm going to pull a sneaky trick on you. Do you want to do great work, or not? Now you have to decide consciously. Sorry about that. I wouldn't have done it to a general audience. But we already know you're interested.

Don't worry about being presumptuous. You don't have to tell anyone. And if it's too hard and you fail, so what? Lots of people have worse problems than that. In fact you'll be lucky if it's the worst problem you have.

Yes, you'll have to work hard. But again, lots of people have to work hard. And if you're working on something you find very interesting, which you necessarily will if you're on the right path, the work will probably feel less burdensome than a lot of your peers'.

The discoveries are out there, waiting to be made. Why not by you?

It's not about just code or academia, it's the core problem i faced with learning and how i learned to learn in a very unconventional way.

But before talking about that i think its worth understanding the conventional way most people learn, so we have a little better understanding on why this unconventional way of learning might be good for you.

The problem with conventional learning

Conventionally we are taught to learn in a linear way. for example

If you wanted to learn web-sockets, in order to learn web-socket you first need to learn about

TCP -> HTTP -> WebSocket upgrade handshake -> web-sockets

this is how you would learn web-sockets. But there are couple of problems with this approach.

• You might drop off in between because the WebSocket upgrade handshake doesn't make any sense
• You skipped one topic in between
• You didn't even know what a protocol is and yet you started from TCP

in short the problem is with understanding. you might have/had less context or you skipped or misunderstood a piece of information without knowing.

If you misunderstood something you will get to know the misunderstanding once you reach web-socket. you will realize how shallow your understanding was about http or maybe about TCP since the beginning and that wastes a lot of time relearning and enforcing new info in the brain.

It happened with me multiple times actually.

The asymmetric encryption

In my college around 2nd year i learned about asymmetric encryption and the concept of public and private key.

if you encrypt a message or information with private key then only private key can decrypt it

but i never understood how you send the keys over from server to client. if you send the private key over the internet to the client then what's the point.

It's a security flaw?

with this misunderstanding i graduated from my college (almost) and when i was having a convo with one of the senior devops engineer and i was understanding how website SSL certs work, asymmetric encryption was one of the key concepts being used there.

when he said we encrypt with private key and send the encrypted message + public key to the client and i was like how the client is going to decrypt it?

He smiled and understood the knowledge gap, explained the whole asymmetric encryption from scratch and then i understood how it works.

in short: you never send the private key over the internet. the server keeps the private key. the public key is meant to be public, you can send it freely.

what messed me up was mixing up encryption and signing. they both use the same keys but do different jobs.

• encrypt for secrecy: encrypt with the recipient's public key, decrypt with their private key
• sign for proof: sign with your private key, verify with your public key. the client is not decrypting a secret message here, they are checking "yeah this really came from the holder of the private key"

SSL/TLS uses asymmetric crypto for authentication and key exchange, then switches to faster symmetric encryption for the actual data. my linear-learning gap was treating "private key" as one thing when signing and encryption are two different jobs.

now you know why linear learning has some flaws. for a lot of people it might be good but for me personally one misunderstood concept or knowledge gap can ruin your whole understanding in the future.

Other issues with linear learning

There are few other issues with linear learning such as forming connections.

You don't know what's the point of learning XYZ thing right now.

But you do it anyways assuming that it will in future connect with some ABC topic and give us the complete picture.

Another issue is that since we are moving linearly, asking questions becomes harder and understanding stays shallow as we progress. you are collecting pieces without knowing how they fit, so you stop asking "wait why?" and just move on.

So now you know the flaws and issues with Linear / Conventional Learning.

The unconventional way

In the unconventional / non-linear way of learning you will learn something like this

WebSockets -> WebSocket upgrade handshake -> HTTP -> TCP -> Protocol

You will start from the topic itself. you will read and as soon as something unfamiliar hits you, you just branch it and go deeper while asking questions.

• Browser
• HTTP Upgrade
• Server
• TCP
• Frames
• Masking
• What is http update?
• What is masking?
• What is XOR masking?
• Why is XOR masking?
• What are frames?
• What frame looks like?
• What is TCP?
• Why we need to upgrade
• Can we not use new protocol?
• Protocol
• XOR Masking

WebSocket information map after reading the definition

As you can see in the above image.

I started reading the definition of websocket and tried forming some connections and did a little bit of branching.

As i was getting new words i was just adding them into canvas and trying to form connections. by the end of the web-socket definition i was left with this.

As soon as this happened i added the arrows (connections) and asked questions on each step mainly on what, why, how.

Information Map

Once the questions are there and a rough diagram is up, i call it an Information Map.

You basically created a map of where you need to traverse now in order to understand web-socket more deeply.

Now all you are left to do is traverse each of the domains in order to concrete the understanding. go deep on the branches that block you, skip the ones that don't matter right now.

That's the whole thing. start at what you care about, map what you don't know, fill the gaps on purpose.

Benefits of non-linear learning

Now i don't think i need to tell you the benefits of this approach. But still

• You can start with whatever interests you
• Just start reading and find the knowledge gaps
• Cover the gaps in order to understand the full picture
• No need to worry about linearity (ofc)
• You catch wrong beliefs earlier because you are forced to connect ideas, not just collect them
• No need to push yourself
• Maps help you choose depth on purpose instead of drifting into random rabbit holes
• You will understand concepts from the core not just superficial

and so on so forth.

I can talk about this a lot.

A foolproof framework (FAFO edition)

Since now we know the pain of linear learning and the benefits of non-linearity, let me tell you that you can convert this into a foolproof system / framework.

This is basically "fuck around and find out" with a map so you don't get lost.

This works for both programming and non-programming things.

The core loop

1. Start with a problem
2. Visualize what you currently think
3. Ask questions
4. Branch based on curiosity
5. Go deep only where needed
6. Read docs / theory
7. Make visual notes (and atomic Obsidian notes if that's your thing)

If you are coding, add this on top

8. Code it
9. Predict before running
10. Tweak / break it
11. Compare prediction vs reality
12. Fix your mental model
13. Explain from memory
14. Connect notes to existing knowledge

Steps 8-11 is the FAFO part. you guess, you run it, you see where reality disagrees, you fix the map.

Tools

You can use excalidraw, tldraw, ipad, tablet for visualization. You can use notion, ticktick, goodnotes, obsidian for atomic notes.

When linear learning still makes sense

I'm not saying linear learning is useless. for exam syllabus, courses, or foundational stuff where order actually matters (basic math before calculus type things), linear can be fine.

The unconventional way shines when you are learning because you need something now, not because a syllabus told you to.

Final Words

That's it for the blog.

Try picking one thing you actually want to learn this week, read the definition, branch every word you can't explain, draw the map, and traverse the gaps. you'll feel the difference pretty fast.

see you in the next one.

Also you can comment below 👇 (it actually works, tag me for faster reply @ramxcodes)

Everything here is a plain text file. There's no database, no login, no admin panel to forget the password to. A post is a file, the file is the truth, and the site is just a nice way of reading the files.

Headings look like this

Body text sits on a narrow measure — about sixty-five characters — because that's roughly the width the eye can track without losing its place at the end of a line. Wider is not more generous. It's just harder to read.

Occasionally you'll want to quote someone. Quotes are set in italic, with a hairline rule instead of a heavy bar.

Lists work as you'd expect:

• Ordinary bullets, tight but not cramped
• Enough room between items to scan
• Nothing decorative

And inline code sits in a soft box, while a longer block gets its own:

npm run dev

Links are underlined faintly and darken when you hover them, so a paragraph full of them still reads as a paragraph rather than a list of buttons.

That's the whole vocabulary. Headings, text, quotes, lists, code, rules, links. It turns out that's almost everything you ever need to write.

It was mid-May. I was a couple of weeks into catching up on post-training literature and learning reinforcement learning for LLMs in a more bottom-up way. Having done some basic experiments, I was unable to come up with an idea that had the allure to possess me for a week or two. I waited as if I was entitled to receive the idea from the muses.

I wondered - if I can be creative in other areas like posting or coming up with ideas when given an objective, why am I not getting good ideas in this specific area of interest. I slowly realised that after years of working on other people's objectives, dismissing building my own projects because they sounded cliche (everyone was doing them); letting small sparks of ideas die because I did not have the energy to protect them, I may have let my idea muscle atrophy.

Don't get me wrong. I do get ideas. Sometimes I get very good ideas and I execute on them for days or weeks on end. It's just not as consistent as I would want it to be. When I say good idea, I don't necessarily mean something novel or groundbreaking. I just mean something that is creative and useful, which can count as a good to great contribution if executed upon well. Maybe it leads to something fun, etc. A good next experiment.

I also realised that in research, coming up with the next good set of ideas, and many of them, matters a lot. I need to start cultivating this muscle better. This led me to contemplate and read more on idea generation/creativity. I am also generally interested in the topic of creativity and since I was on a break, it was a thing to do.

Moreover, intelligence is commoditised (at least in knowledge and coding work due to AI). Agency and taste are more important than ever (I think we have consensus on this now?). Good ideas are the next great adventure. After all, without a good idea what use is agency?

So now here we are - writing a post to record my observations and learnings. Making my problem everyone's problem. I also write this in the hope that some of you (and future me) will execute on my good ideas better than me.

The basics and where we go wrong

The folk recipe for idea generation is simple - do consumption-maxxing, marinate on it for days, walk around, get caffeinated, yap with friends over it and wait for the muse to whisper. I wondered if it could be a bit more systematic? Did the muse have an API? Can I at least learn how to bounce back from dull states to generative states? This was my "struggle". Most of this post is my attempt at making it a bit more systematic.

Most of us do the mistake of consuming in excess and call it thinking. We don't spend enough time processing or applying new information. Perhaps I was more study heavy too than code/experiment heavy (narrator voice: indeed he was.). I liked this particular quote -

After you read something, watch something, listen to something — before you look at what anyone else thinks about it — write down what you actually thought. Not what you think you're supposed to think. Not the smart take. Write down what you actually noticed, what bothered you, what you believed before and whether it changed. Source

Maybe I should consider doing an exercise regularly where I get away from screens and music and just try to come up and write down ideas on paper for 5 minutes. Maybe I am not taking out time to think on things, rushing from one thing to another and constantly context switching.

Good inputs and direct exposure to the source

LLMs have made it dangerously easy to live one layer above the source material. People are not reading code directly, using AI to write and even use AI to read through long content. If you want to get good ideas and you are a serious people (Logan.jpeg), then you need to go through the content you have selected to go through yourself. Then talk about / think about it a bit. On a meta level, the idea is to consume good inputs in high resolution to build a good embedding space of knowledge. If you have a good embedding space, it will eventually light up when you come across a relevant problem statement and lead to a generative state.

Discovery happens in the doing

Common thing we mess up is waiting too long before applying the newly learnt inputs (or ideas). You have to get to applying your ideas as early as possible, even the silly ones. Get into a feedback loop. This reveals gaps in your understanding and feeds into your thinking better. Once you start, the work begins to feel like a quest. One experiment leads to another. You run into new problems, notice new angles, and sometimes find people who care about the same things, especially if you talk about your work online and offline.

You will hear AI researchers reproduce papers or (even engineers) trying to at least get things running. This helps to identify the details that were not mentioned in source material and also finding gaps in your own understanding.

You have to do a lot and then consume a lot. A common mistake is being consumption heavy and too light on doing. Another mistake is that if you get into doing and are making progress, you may get tunnel visioned by doing and delay learning relevant things that you should be following up on. There is a sweet spot and I think you should definitely lean towards doing. Doing and getting feedback from tight loops is more psychoactive in general. Doing/creating leads to more feedback from reality, people and subsequently more mini problem statements. Sometimes, it may help you discover who you are.

Do more of things that make you feel alive. Follow the things that give you energy after you do them for sometime. Some activities drain you while others return you to yourself.

It's a matter of time (we often underestimate the effort)

Idea generation in its pure essence is a matter of putting in the time. In my case, the answer was less mystical. I was simply early into the inputs for learning RL. Another thing is I was approaching it in a bottom up way. I was learning and just needed to put more time in it in terms of doing more basic experiments; Accept that I am still a fool in this and I need to be humble enough to struggle and learn from basic things, play more with the code/experiments. This whole process is inefficient and can't be speedrun through.

It's similar to how you need to put in some hours in an RPG like Dark Souls 3 or Elden Ring till your build starts to mature and you find the better weapons. Then you start having fun, with a better build and understanding of the game mechanics. Some part of your brain gets freed up from learning for survival and now you have bandwidth for creativity.

Coming back to RL learning, I made 2 mistakes here: I was not humble enough and I was trying to do a speedrun bottom up. If you are attempting a learning speedrun, be more top down oriented or objective oriented.

In general, I find it better to read things when I actually need them, and that usually works for me. The RL learning was a bit of an exception. I started to enjoy the reading, especially going from the PPO and GRPO basics to the latest on-policy distillation literature and then reading a couple of new papers. I also had to refresh a lot of maths, and I figured I could afford to since I was on a break.

Normally I lean top-down, especially now with LLMs around. You get a lot done fast that way, but the learning ends up feeling fragmented. I am sure there is a sweet spot somewhere. Like I mentioned earlier, it is easy to get tunnel-visioned by doing and skip the reading you should be catching up on. The sweet spot is staying disciplined enough to actually read and understand the material, and to take pauses. I am expecting that when you read stuff while in the loop, you are more likely to get ideas too.

Ideas come from inside the work

Another thing I learnt which is in the line of "doing things" is that you are more likely to get ideas when you are inside the work. It's like being in the mine and finding precious stones covered in the mud and dirt. The gaps, bottlenecks, missing details, things you encounter while solving blockers, what if I had done X way, was there a better way to do this?

Some attempts to prose aside, this means you are in the loop. The inputs are freshed and processed. You have recently wrestled with the problem. If you are not currently fresh with the inputs and worrying about getting an idea, you are wasting your time. When you are out of the loop, relax. When you will get back, the wheel should get started.

The people are algo driven and the algo is people driven too

I post a lot on X. When it comes to posting, I have been pretty creative often especially when I have had some momentum of posting for few days consistently. However even there, a lot of my posts are inspired from the discourse (which objectively is not bad and something that I don't mind until it gets boring). Such is the nature of posting of most people. It's influenced by current algo, current discourse. Both the people and the algo are to blame for it because people are more likely to like what's familiar and the algo would subsequently push more of it. From a meta perspective, the discourse is a scaffold, or you can say a prompt.

This does not just happen in posting, people will write articles, blogs, even make projects based on discourse. High agency college students are very good at it (it's a good thing to have). People will build startups, in fact most of the startups probably start this way, either some interesting in discourse (Karpathy tweets about auto-research and 10 auto-research startups come out of stealth). Cursor gains traction or Claude Code gets some success, 5 other startups with 1-2 unique variants pop up.

I have talked to several researchers and engineers and they would often mention that they got inspiration to write a blog or make a project because they thought how a certain product they use works or what would happen if they tweaked x in this product or how can I vary this to solve this problem. They got interested in it and executed upon it within their capability. The meta here is curiosity, pro-activeness to try the tools and lastly the whole activity acting as a scaffold/prompt.

There is one interesting thread here that how do you decouple your ideas from the discourse/algo and get more out of distribution ideas. I will refrain to get into it for this post.

You need a prompt

Ok the point I am arriving to is you need a prompt/scaffold. Ideas don't just pop out of nowhere. You need something against which you can bounce off your mind's embeddings, your mind's substrate such that something in there activates and you get sparks. You need to have a prompt, a scaffold, an objective, a problem statement. The mind needs something specific to push against before it gives you anything back. Ideas need friction. You are not entitled to ideas; you have to work for them.

Something for your mind to push and pull against. The problem statement could be as simple as "I want to make more money" and then you may have ideas downstream of that. The mind has this tendency to develop phosphorescence once you get serious about something. It should be based on what you really want to do. I am tempted to say the more specific the better it is, however specificity leads to the trap of being tunnel visioned. Higher resolution would be a better choice.

Agency is downstream of wanting. To what extent you care about it. The more something matters to you, the more your mind recruits the world around it: people, tools, problems, ideas, coincidences.

Post that, it's just a matter of being in a state where you are sensitive enough (not being overstimulated, long walks, caffeinated etc. whatever works for you) to hear it. This is more self-experimentation driven. What routine, foods, people, activities help you to become more generative. Having good health and routine. Maybe having a side quest along with main quest. When I am in such a state, more ideas come. I talk to more people and do more, and it compounds. This is the closest thing to a system I have found.

I recommend reading Henrik Karlsson's Good ideas and this video about the systems and routines of writers/directors like David Fincher. They tend to have strict routines of creative work. David has this thing where he would try to keep as much order as possible in his day such that his mind has space to think of chaos when he sits to imagine and write.

a problem statement is all you need

I get a lot of ideas when I am strongly pursuing a problem statement (after spending lots of time reading or just building with an agent or trying to solve a specific blocker). In fact this is one of my specialisations: to go unreasonably deep into a particular trench if I have to solve something.

When you are trying to solve a problem or attain an objective for whatever reason, you end up doing pretty much everything I have talked about so far.

when you don't have a prompt, just do things

Sometimes you won't have that prompt/objective/scaffold and in those cases you will have to just keep doing and consuming. Doing things may lead you to discover "goals" you want more than your preconceived goals / the goals your rational mind formulated. Expose yourself to good and diverse inputs, let yourself indulge in your curiosities at times. Consume and do things in the direction that makes you feel alive.

Screenshot 2026-06-29 at 3

Last month, I was in a funk. I was not excited about anything I used to be excited about. I was recovering from a mild illness and had been out of touch with things that I do regularly. He had suggested to do the silliest thing and you may find aliveness in it (and it kinda worked). I wrote a blog on my favourite kpop songs and got my "doing" momentum back.

people are like portals and mirrors and good conversations can re-arrange your brain (h/t Maja)

Talking to people helps a lot. Bouncing back on each other's existing and new ideas. Letting them ask questions, asking them questions. Try to think from their perspective. If you have a few friends who are familiar with your context, they can often remind you what it's like to be the creative you and that can be a great fuel. I am lucky to have few such friends.

Good conversations with good people can help you to spot blind spots. You can also spread context about yourself so others can help you and you can help them in times of need. You can also collab with others, work with others and get ideas from there. Recommended reading: why some conversations rearrange your brain

Below are couple of quotes on “Keeping your door open” from Gwern’s transcript of Richard Hamming’s “Your and Your Research” talk. On the importance of being open to interact with other people to understand what advancements are happening in the world and what problems you should be working on.

Another trait, it took me a while to notice. I noticed the following facts about people who work with the door open or the door closed [cf. "working with the garage door up"]. I notice that if you have the door to your office closed, you get more work done today and tomorrow, and you are more productive than most. But 10 years later somehow you don't quite know what problems are worth working on; all the hard work you do is sort of tangential in importance. He who works with the door open gets all kinds of interruptions, but he also occasionally gets clues as to what the world is and what might be important.

Now I cannot prove the cause and effect sequence because you might say, "The closed door is symbolic of a closed mind." I don't know. But I can say there is a pretty good correlation between those who work with the doors open and those who ultimately do important things, although people who work with doors closed often work harder. Somehow they seem to work on slightly the wrong thing—not much, but enough that they miss fame. Richard Hamming, You and Your Research

Introspect by doing

I love this entry from Henrik Karlsson's blog Two kinds of introspection where he nudges us to introspect by doing. To let the doing reveal the world in us.

When Cave sits down each day from 9 to 5:30 pm—he’s a punctual man—hammering out lyrics, listening for the “supreme spiritual potential that only comes into its true and full being if we abandon all those cherished ideas about who we think we are or are not”, he pretends to be battling with the muses. This is the game he plays with himself. But there are no muses. The voice he hears at his typewriter when he listens carefully and blocks out the opinions of others is the voice of Nick Cave pretending to be the possessive god of art.

He is introspecting, but he’s not introspecting by meditating on his existence or talking to a therapist (though maybe he should); he’s introspecting by acting on the world. He’s introspecting by doing stuff and then observing how it makes him feel—as if he was running a million small experiments trying to figure out the composition of this object that is Nick Cave. He is ascribing the feelings he is hearing to the muses, and that might be a good way to think about it as an artist, but the feelings are his. When the muses tell him to lean into a song, to expand it, it is his taste and experience talking.

Screenshot 2026-06-29 at 1

Breaking through a block

Changing gears a bit, I have focused on "initial ideas" so far but I think most of it transfers for when you are facing a creative block or just blocked at something with respect to problem. In such situations, taking a break helps. Talking, listing out your assumptions and questioning them helps. Simplify the approach. Reread the problem statement. Reframe what you think you are solving. Often the block is not in the problem, but in the cached shape your mind has given it.

I think the main reason I didn’t realize my problems were solvable was a lack of imagination. I’d never seen anyone solve problems in an agentic way, and I failed to imagine that things I hadn’t seen done could be done by me. Source

Taking a break and long walks work because it allows to free up your mental context and zoom out. You can later see things from a fresh perspective, with maybe looser constraints. You often need to loosen up constraints and slither out of the invisible binds on you. Try to be playful. Personally, I do better when I don't take myself too seriously.

If you have a lot of freedom and not many external motivators like a deadline, then imposing certain constraints may help.

Besides this, now that we use AI agents a lot, we often don't have a full understanding of the system. Consider increasing your resolution by going below the level of abstraction you are currently at, i.e. descend into the data/code/log trenches yourself and sweep the solution search space with help of agent.

Screenshot 2026-06-29 at 2 — Source: Jack Morris's Zen and the Art of Machine Learning

There will be times when you are just genuinely blocked and can't really do anything. I asked one of my AI researcher friends at a big lab about it and they mentioned that they usually have multiple projects going on so if one gets hard-blocked on one project, pick up another. They suggested me to consider increasing breadth when learning something new (study it at different levels of resolution/abstraction, study the foundations and the application etc.).

Things I enjoyed reading

A few things I found helpful or just enjoyed while thinking about all this.

On AI research and engineering

These are more AI research specific and very good reads.

I highly recommend reading Jack's post. This post is partly inspired by it. Another one I had enjoyed recently was Karina Nguyen's post.

Few quotes that had resonated with me:

The gap between “trying random things” and “systematically narrowing hypotheses” separates okay researchers from effective ones. Debugging intuition is learnable and comes with running hundreds of experiments.

The most durable skill in AI research isn’t any particular technical contribution, but the ability to repeatedly identify what matters now and execute quickly. Choosing the right problems to work on is as important as executing them with care.

Some discoveries are downstream of infrastructure—you can’t have certain ideas until you have the systems that make them possible to test. The tooling comes first, then the insight. That’s why often researchers spend time on engineering problems.

On creativity, agency, and people

None of us are stuck because we have no options. We’re stuck because the alternative has stopped being something we can picture for ourselves. And in that gap, many of us lose the ability to imagine different futures at all. Source

"If you reject your own ideas, then the part of the brain that comes up with ideas is going to stop. You just do it and do it and do it, and you sort it out later."

Screenshot 2026-06-29 at 2

I loved this entry on Riley Walz where he talks about ideas and people; it was an inspiration for this post. He is into it for the fun. When people see him dance on his ideas, they join the dance too.