Skip to content
>dp
← journal

A Request Walks Into A Spring Boot App

One GET, traced from the socket to the database and back, past the threads it borrows and the proxies it passes through, to the boring part where all the milliseconds live.

9 min read

#spring#java#jvm#databases#performance

GET /books/1 is the most boring request I can think of. One row, one JSON object, no auth, no cache, no fan-out. On a warm JVM it comes back in about a millisecond.

In that millisecond it is handed between at least six pieces of machinery that most of us never wrote and rarely read. Every one of them has a queue. Every one of those queues has a default. And when a Spring Boot service falls over at 3am, it is almost never the controller. It is one of those defaults meeting a load nobody sized it for.

So let's follow the request. Not the sanitised three-box version, the actual route.

CLIENTTOMCATNIOACCEPT + PARSEFILTERCHAINSECURITY, ENCODINGDISPATCHERSERVLETROUTE + BIND@CONTROLLERYOUR CODEDATABASEONE ROUND TRIPHIKARICPPOOLBORROW CONN@TRANSACTIONALBEGIN TX
The round trip, wrapping to a second row that runs right to left. Arrows give the direction of travel; the filled dot is the request and the hollow one is the response, retracing it exactly.

The Part Before Java

Your process is not listening in any interesting sense. The kernel is. Tomcat's NIO connector runs a small acceptor thread whose entire job is to pull finished TCP connections off the accept queue and hand them to a poller. The poller watches sockets for readable bytes. Neither of them will ever execute a line of your code.

Three numbers govern this stage, and they are the ones people discover last:

SettingDefaultWhat happens when you hit it
server.tomcat.max-connections8192The acceptor stops accepting
server.tomcat.accept-count100The OS backlog fills; new connections are refused
server.tomcat.threads.max200Requests wait for a worker, holding a connection

That middle row is the one that produces the confusing incident. Your service is up, health checks are green, the JVM is not even warm, and clients get connection refused. The app never saw those requests. They died in a queue 100 deep.

Your Request Gets A Thread, And Keeps It

When bytes for a full request line and headers have arrived, the poller hands the socket to a worker from the thread pool, and you get a log line stamped http-nio-8080-exec-7. That thread now belongs to your request until the response is written. All of it. Including the part where you are doing nothing but waiting for Postgres.

That is the whole servlet model, and the arithmetic is blunt: 200 worker threads and a 50ms database call means a ceiling of roughly 4,000 requests per second, no matter how fast your code is.

SOCKETFILTERSROUTEYOURCODEWAITING ON THE DATABASETHE THREAD IS DOING NOTHING AT ALL HERESERIALISEHTTP-NIO-8080-EXEC-7 — HELD FOR EVERY MILLISECOND OF THE ABOVE
One request, one worker thread. The wide segment is where the thread does nothing at all, and it is drawn narrower than it really is, so the dot lingers there for less of the loop than it would in production.

Virtual threads are the interesting answer to this, and in Boot they are one line:

spring.threads.virtual.enabled=true

The blocking style of the code above does not change at all. What changes is who pays for the blocking: a virtual thread parked on a socket read costs a few hundred bytes of heap instead of a megabyte of stack, and the carrier thread goes off to run something else. The 200-thread ceiling stops being the constraint. Your connection pool becomes the constraint instead, which we will get to, because it always does.

The Filter Chain, Where Exceptions Go To Hide

Before Spring MVC sees anything, the request passes through the servlet filter chain. Character encoding, security, request logging, whatever you registered. Filters wrap each other like nested function calls: each one gets to run code on the way in, call chain.doFilter(...), and run code on the way out.

There is a trap here that has cost me an afternoon more than once. @ExceptionHandler and @ControllerAdvice are Spring MVC machinery, and Spring MVC lives inside the DispatcherServlet. An exception thrown in a filter is outside that box, so none of your handlers run. What you get instead is the container's error dispatch and Boot's default /error response, which is why an authentication failure sometimes returns a shape your API contract has never seen.

DispatcherServlet: The Front Controller

Now the request reaches Spring. DispatcherServlet.doDispatch is roughly a hundred lines long and explains the entire framework. In order:

  1. Find a handler. Each HandlerMapping is asked in turn. RequestMappingHandlerMapping holds the map built at startup by scanning every @RequestMapping in the context. Matching happens against a prepared structure, not by walking your controllers per request.
  2. Get an adapter. The handler is a Method plus a bean, not something the servlet knows how to call, so a HandlerAdapter is chosen to invoke it.
  3. Resolve the arguments. This is where @PathVariable long id becomes 1L. A chain of HandlerMethodArgumentResolvers each claim the parameters they recognise. Same mechanism binds @RequestBody, @RequestParam, Principal, Pageable.
  4. Invoke your method. Reflection.
  5. Handle the return value. A HandlerMethodReturnValueHandler sees ResponseEntity<Book>, notices @ResponseBody semantics, and picks an HttpMessageConverter by content negotiation.

Nothing about that pipeline is magic, but all of it is indirection, and the indirection is the point: every step is a list you can add to.

Your Method, And The Proxy You Did Not Write

The controller calls the service. Except it does not, at least not directly. If the service is @Transactional, what the controller holds is a proxy, and the proxy is where the transaction begins.

Two consequences follow.

The first is the classic bug: a @Transactional method calling another method on the same object does not go through the proxy, so the second annotation does nothing. No warning, no error, just no transaction.

The second is quieter and worse. With DataSourceTransactionManager, the connection is borrowed from the pool when the transaction begins, not when you run your first query. So this holds a pooled connection for 200ms while talking to a service that has nothing to do with the database:

@Transactional
public Order place(OrderRequest request) {
  // 200ms, and it holds a pooled connection
  var quote = pricingClient.quote(request);
  // the only line that actually needs one
  return orders.save(new Order(quote));
}

Under load that is not a slow endpoint. That is a pool exhaustion incident with a misleading stack trace.

The Pool Is Smaller Than You Think

HikariCP's default maximum pool size is 10. Ten. Meanwhile Tomcat is handing out 200 threads. The funnel is not an accident and it is not too small. Hikari's own sizing guide argues at length that small pools are faster, because a database with eight cores does not go faster when 200 connections compete for them.

ACCEPTBORROWACCEPTEDCONNECTIONS8192WORKERTHREADS2003 THREADS WAITINGCONNECTIONS — 10DATABASEEVERY ONE OF THESE IS A QUEUE WITH A DEFAULT
Three queues in series, each an order of magnitude smaller than the last. The count above the pool is computed from the arrival schedule; widening anything to the left of it only moves the queue.

When the eleventh thread asks for a connection it waits, and it waits for spring.datasource.hikari.connection-timeout, 30 seconds by default, before failing. Thirty seconds is an eternity to a caller with a 2-second timeout. This is why a slow query does not degrade your service gracefully; it converts a database problem into a thread-pool problem one layer up, and then into refused connections one layer above that. Queues in series fail backwards.

The Database, Briefly

JdbcTemplate takes the bound connection, prepares the statement, binds 1L, and writes it to a socket. Then everyone waits for the network, the planner, a page that is hopefully in the buffer cache, and the network again. Rows come back as a ResultSet, and your RowMapper turns cursor positions into a Book.

This is the only part of the whole journey doing the thing the user actually asked for, and on a healthy service it dominates the wall clock. Everything else in this article is overhead: well-organised, extremely useful overhead, but overhead.

The Way Back

The return path is the same picture in reverse, with one detail that catches people.

The transaction commits and the connection goes back to the pool when your service method returns, before the JSON is written. Serialisation happens afterwards, out in DispatcherServlet, with no database connection in hand.

So what happens when you return a JPA entity with lazy associations? Not the exception most people expect. Spring Boot leaves Open Session In View on by default, so the Hibernate session outlives your service method and Jackson can still initialise whatever it walks into. Instead of LazyInitializationException you get a fresh query, on a freshly borrowed connection, for every association it touches, at the point in the request where you thought the database was behind you. The exception is what you see once you turn that default off, which is a later problem and a better one.

Then the converter writes bytes into the response buffer, Tomcat flushes them to the socket, the worker thread is returned to its pool, and the connection stays open for the next request on it. Your millisecond is over.

Where To Actually Look

If you take one thing from the shape of this, take the order to check things in when latency goes bad, because it is roughly the reverse of where people start looking:

  • The pool before the code. hikaricp.connections.pending and hikaricp.connections.acquire tell you in seconds whether you have a database problem or a Java problem.
  • The transaction boundary before the query. A fast query inside a long @Transactional method costs the same as a slow one.
  • The queue before the thread. If tomcat.threads.busy is pinned at max, your p99 is queueing time and profiling the controller will show you nothing.
  • The controller last. It is almost never the controller.

Spring Boot is not slow, and your code is probably not slow either. What is slow is waiting. Almost every layer on this trip has a queue in it, a default somebody else chose, and a metric that will tell you so. Knowing where those three things live is most of performance work.

References

Version-specific details were taken from a Spring Boot 4.1.0 scaffold on Java 25 (Tomcat 11.0.22, HikariCP 7.0.2; note that Boot 4 adds spring-boot-starter-webmvc and deprecates spring-boot-starter-web, which still resolves). Behavioural claims come from: