Muhammed Senussi
Muhammed Senussi
  • Muhammed Senussi
Virtual Threads in Anger: Six Months On
Java

We moved three services onto virtual threads last winter. Throughput barely moved on two of them, which surprised the team until we looked at where time was actually going.

The win is not speed

A virtual thread costs a few hundred bytes instead of a megabyte of stack. That does not make any single request faster. What it changes is that blocking stops being the thing you design around, and the reactive machinery you adopted to avoid it stops paying for itself.

If the only reason a service is reactive is that platform threads were expensive, virtual threads have removed the reason.

What actually bit us

  • synchronized pins the carrier thread. We found two hot paths doing exactly that under a library we did not own.
  • Pooling virtual threads is an anti-pattern. Create one per task and let them be cheap.
  • ThreadLocal still works but scoped values fit the model better, especially for request context.

The migration was mostly deletion: swap the executor, hunt for pinning, and remove the operators that existed only to avoid blocking.

4 Comments

  • Peter Bakker

    February 3, 2024

    The pinning point matches our experience exactly. How did you find the two hot paths — JFR?

    • Muhammed Senussi

      Muhammed Senussi

      AuthorFebruary 3, 2024

      JFR, yes. The jdk.VirtualThreadPinned event gives you a stack trace per pin. We ran it for an hour under normal load and both offenders showed up in the first minute.

  • Yara Mansour

    February 5, 2024

    Did you consider staying reactive for the streaming endpoints?

    • Muhammed Senussi

      Muhammed Senussi

      AuthorFebruary 5, 2024

      We did, and we kept one. Backpressure over a long-lived stream is still something reactive expresses better than a thread per connection. It was the request/response services that had nothing left to gain.

Leave a comment