All of Adrian Kelly's Comments + Replies

They claim that by specializing the chips for transformer workloads and removing the programmability of GPUs, they can fit an order of magnitude more compute FLOPs on the same size chip, which is plausible. But common wisdom is that LLMs are memory bandwidth limited. Model Bandwidth Utilization in inference workloads is often 60-80%, which would indicate that Nvidia's chips are reasonably well balanced in their ratio of bandwidth to compute, and that here isn't a ton of performance to be gained by just increasing compute. The Sohu chip reportedly has 144GB... (read more)

2Tao Lin
the reason why etched was less bandwidth limited is they traded latency for throughput by batching prompts and completions together. Gpus could also do that but they don't to improve latency

IBKR just announced a new prediction market, and it pays interest on the value of your positions (fed funds rate minus 0.5%)

Domain: Engineering

Building Prototypes with Dan Gelbart https://www.youtube.com/watch?v=xMP_AfiNlX4&list=PLSGA1wWSdWaTXNhz_YkoPADUUmF1L5x2F&index=1 

Dan Gelbart has been Founder and CTO of hardware companies for over 40 years, and shares his deep knowledge of tips and tricks for fast, efficient, and accurate mechanical fabrication. He covers a variety of tools, materials, and techniques that are extremely valuable to have in your toolbox.

1Parker Conley
Thanks! Added.

Yeah swaptions would be nice but it seems like the minimum size is $1mm.

Why not just short-sell treasuries (e.g. TLT)?

Futures and options give you a lot more leverage than short selling. A $100k short position on TLT would be $30k of maintenance margin, compared to $7,400 for UB.

And banks and hedge funds arbitrage futures prices against the underlying asset, so trading futures basically gives you access to institutional interest rates instead of retail margin rates. Right now the rate difference for short selling on IBKR is ~5% for accounts <$100k and 1... (read more)

I spent a couple hours looking at different methods to efficiently short long term bonds:

  • UB Treasury Bond Futures - 30 year bonds but you have to roll every quarter on the roll date which is both a hassle and you pay the spread each time you roll. Also, the expected return if the world stays normal is significantly negative, it should be the 30 year rate minus the risk free rate, for which the average since 1977 has been 2% per year.
  • SOFR Futures - pays out based on the average interest rate in a specific 3 month time period up to 10 years out, though liqui
... (read more)
2Jonas V
Very helpful, thanks. And interesting about the Eris SOFR Swap Futures.  Interest rate swaptions might also be worth looking into, though they may only be available to large institutional investors. Why not just short-sell treasuries (e.g. TLT)?