Startup Ideas Bank
An interpretability tool that's more of a hobby project than a product.
AI roast score: 35/100 (F)
The idea
anthropics/jacobian-lens — Companion code for the global workspace interpretability paper
jlens — Jacobian lens
Reference implementation. Not maintained and not accepting contributions.
Companion code for Verbalizable Representations Form a Global Workspace in
Language Models .
The Jacobian lens reads out what an internal activation is disposed to make the
model say. It linearly transports a residual-stream vector at any layer and
position into the final-layer basis, then decodes it with the model's own
unembedding into a ranked list of vocabulary tokens.
The transport is the average input–output Jacobian over a text corpus:
lens_l(h) = unembed( J_l @ h ), J_l = E[∂h_final / ∂h_l]
The expectation is over prompts, source positions, and all current-and-future
target positions in a generic web-text corpus; the precise estimator
(cotangents summed over target positions, then averaged over source positions)
is documented in the jlens.fitting module docstring.
This repo fits the lens on open-weights decoder transformers, applies it, and
renders the interactive layer × position view shown below. Examples use Qwen;
other HuggingFace decoders adapt cleanly.
The ASCII-face example: selecting the ^ (nose) position shows the lens
reading out "nose" at mid layers, although the word never appears in the
prompt.
Install
pip install -e .
Usage
Apply
To apply a pre-fitted lens:
import transformers , jlens
hf = transformers . AutoModelForCausalLM . from_pretrained ( "org/model" ). cuda ()
tok = transformers . AutoTokenizer . from_pretrained ( "org/model" )
model = jlens . from_hf ( hf , tok )
lens = jlens . JacobianLens . from_pretrained ( "org/lens-repo" , filename = "model/lens.pt" )
lens_logits , model_logits , _ = lens . apply (
model , "Fact: The currency used in the country shaped like a boot is" ,
positions = [ - 2 ])
for layer , logits in sorted ( lens_logits . items ()):
print ( layer , [ tok . decode ([ t ]) for t in logits [ 0 ]. topk ( 5 ). indices ])
Fit
To fit a lens on your own model:
lens = jlens . fit ( model , prompts = my_prompts , checkpoint_path = "out/ckpt.pt" )
lens . save ( "out/jacobian_lens.pt" )
The paper's lenses use 1000 sequences of 128 tokens from a pretraining-like
corpus. Quality saturates quickly (§9.3); ~100 prompts is usable. This is a
reference implementation and is not optimized; fitting time is dominated by
the model's own backward pass. Parallelize by running fit() on disjoint
slices and combining with JacobianLens.merge() .
Walkthrough
walkthrough.ipynb is the end-to-end notebook: load a
model, load (or fit) a lens, apply it at a few layers, and render a slice page
like the one above.
Reading a slice page:
Each cell shows the lens top-1 word at that (position, lay
The roast
This project has all the hallmarks of a weekend hobby that's nowhere close to being a viable business. It's unmaintained, not accepting contributions, and the founder hasn't even decided on a revenue model. The idea might intrigue a handful of AI researchers, but that’s a far cry from a scalable business. Furthermore, the solo founder with no funding and an undeveloped moat is a glaring red flag for anyone considering this as a serious venture.
Your biggest unknown is whether anyone will pay for this, and given the lack of a pricing model and deal size over $10, the answer is likely no. Without a dedicated team or any form of capital, you have no runway to even experiment with monetization strategies. The global geography of your intended market is irrelevant if you can't even get local traction.
In summary, you're offering a complex but niche tool that doesn't solve a broad or urgent problem. Your execution plan is non-existent, and without a clear path to revenue, this is more of a research artifact than a business.
Red flags
- Unmaintained project
- No revenue model
- Solo founder with no funding
Verdict
This is a niche research tool lacking the fundamentals to become a business.
Roast your own startup idea →