Simpler Machines

A close read of the ActionDispatch::TestResponse class

Nat Bennett -

Last week I talked about how I didn't know what an ActionDispatch::TestResponse was, or how and why it gets returned in tests instead of an ActionDispatch::Response. This week we'll figure that out with a close read of the code for the TestResponse class. Along the way, we'll learn about "magic comments" that provide directives to the Ruby interpreter, what exactly a MIME type is, and a little bit about lambdas.

Anyway! Let's take a look at that code, shall we?

# frozen_string_literal: true

require "action_dispatch/testing/request_encoder"

module ActionDispatch
  # Integration test methods such as ActionDispatch::Integration::Session#get
  # and ActionDispatch::Integration::Session#post return objects of class
  # TestResponse, which represent the HTTP response results of the requested
  # controller actions.
  #
  # See Response for more information on controller response objects.
  class TestResponse < Response
    def self.from_response(response)
      new response.status, response.headers, response.body
    end

    def parsed_body
      @parsed_body ||= response_parser.call(body)
    end

    def response_parser
      @response_parser ||= RequestEncoder.parser(media_type)
    end
  end
end

Right away, a mystery:

1 # frozen_string_literal: true

Allegedly, a hash at the start of a line marks that line as a comment, but this looks like code. Weird code. Possibly directions to some kind of pre-processor or build thingy or something, possibly just code that should have been removed that's still hanging around.

A quick look at git blame reveals the commit that introduced that line, which isn't as enlightening as I'd hope. The commit just says what the commit does, not why. This probably isn't relevant to what we're trying to understand about the code, but I'm curious, so I'll google "frozen_string_literal."

This is enlightening: A Stack overflow answer that exactly describes my question. From this I learn

  • There's something called a "magic comment" that provides directions to the Ruby interpreter
  • This one makes all strings immutable. Handy!
  • Making strings immutable increases application performance by reducing garbage collection chores, and prevents other code from modifying strings that we know should never be modified
  • This was going to be the default behavior in Ruby 3.0, but Matz backed off due to the potential compatibility issues

Having watched the Python3 mess from afar, I pause for a moment of gratitude.

I've never heard of a "magic comment" so let's go look up what that is while we're here.

Here we learn a few more things

  • There are a few others, but not many
  • Their primary use for controlling the default encoding of string literals
  • There's also a indentation warning
  • Ruby 3.0 introduces a new one
  • There's a link to the full list of magic comments in the Ruby source code

(Incidentally – most of what I know about C I've picked up from reading Ruby source code, and my C knowledge is mostly useful for understanding why Ruby builds failed.)

Well that was well worth the ten minutes or so it took, I think! I return to the file and see... no string literals. Hmm. Well. Figuring out whether that comment actually does anything here can be left for another time, I think.

2 
3 require "action_dispatch/testing/request_encoder"
4

Since we're binding late that tells us less than one might hope about what might be in our execution environment once this code is actually running. Still, handy to know that we're pulling this in. Let's take a quick look at the request encoder.

A bunch of request properties... and some stuff about MIME types. I vaguely remember those as a thing that can cause e-mails to look weird if your e-mail client isn't fancy enough, so let's go look 'em up.

  • It stands for Multipurpose Internet Mail Extensions
  • It's kind of like a file extension, but for resources on the internet
  • There's no complete list of all of them but some common ones have been documented by Mozilla

Ah, so this is what that "application/json" header business means. Cool!

Back to the TestResponse

5  module ActionDispatch
6    # Integration test methods such as ActionDispatch::Integration::Session#get
7    # and ActionDispatch::Integration::Session#post return objects of class
8    # TestResponse, which represent the HTTP response results of the requested
9    # controller actions.
10   #
11   # See Response for more information on controller response objects.
12   class TestResponse < Response

Now we're getting somewhere!

This is a subclass of the regular ActionDispatch::Response that adds some new behavior. Makes sense. I'll take a quick look at the ActionDispatch::Response code to see if there's anything I need to know there.

  • Represents a Response in what looks like a pretty classic OO way, it has fields for all the various properties and relationships of a response.
  • It's a Rails implementation detail, so I should never see it in production code. I might see it in tests.
  • It's created with a status 200 and empty headers and body by default.
  • That MIME type is set by a method called content_type

The one thing that surprises me is that we call super() and super in a couple of places, and there's no obvious parent class. I check git blame and find a commit that I don't understand at all.

I'm tempted, at this point, to start digging around in my local copy of the Rails gem, but since we're now 800 words into this and haven't gotten to anything that seems relevant to what I'm trying to understand, I'm going to make a note to take a look at this later and move on for now.

13    def self.from_response(response)
14      new response.status, response.headers, response.body
15    end

Our first method! There's no equivalent in the parent class.

I'm not sure I could describe this in plain English. We're... calling constructors? Is this line equivalent to...?

response.status.new, response.headers.new, response.body.new

Maybe looking at how this method gets called will help.

Looks like this is called once and only once in Rails. Huh, this looks vaguely familiar: ActionDispatch::Integration::RequestHelpers::Session.process Okay, yeah, the syntax is mildly novel but I'm looking at... oh. Oh, I see. Right. Because everything is an object, new bare means CurrentObject.new, so if we go back to the parent class...

def initialize(status=200,header={},body=[])

Okay, so this is a wrapper around our own initializer so we can unpack a Response and make a TestResponse without the caller having to know what to unpack.

Next method, then...

17    def parsed_body
18      @parsed_body ||= response_parser.call(body)
19    end

I recognize the ||= syntax as memoization, this is saying, "return the instance variable @parsed_body unless it doesn't exist yet, in which case call response_parse.call(body), assign that to @parsed_body, and then return it.

What the heck is a response_parser, though? Hopefully it's in that request_encoder thingy we're requiring... yeah. There's an attr_reader and when the request_encoder gets initialized...

@response_parser = response_parser || -> body { body }

Errrrrr.

Okay, let's break this down.

@response_parser = response_parser ||

Up until this point, no problem. We're setting the value of an instance variable, and we're setting it to either response_parser if response_parser is truthy, or whatever comes after ||

->

I think this has something to do with... hash assignments?

Luckily it's easier to Google than I thought it was going to be, and this is a lambda literal!

Which means... um. Oh. Okay. Had to think about that for a minute but it means that if we don't already have a response_parser we'll assign... oh. We'll use a function that just hands back whatever it was given.

Right, because we're in the initializer for request_encoder. So what this is saying is, "if you don't get passed a response_parser, don't parse responses." I wonder under what circumstances we do anything else, and where this gets new'd up, but that's probably a question for another time.

Oh hey though, that question gets answered immediately, because this isn't request_encoder's response_parser, it's ours.

21    def response_parser
22      @response_parser ||= RequestEncoder.parser(media_type)
23    end

Another straightforward instance of memoization. Checking back with the request_encoder we see

    def self.parser(content_type)
      type = Mime::Type.lookup(content_type).ref if content_type
      encoder(type).response_parser
    end

So if content_type got passed in, we look it up in what's presumably a table of Mime Types or something, get this ref thing, and set it to type. Then we pass that or nothing into encoder and call response_parser... and encoder is another request_encoder method which... either returns the encoder named by that type ref, or a default @encoders[:identity]

    def self.encoder(name)
      @encoders[name] || @encoders[:identity]
    end

Great, so, let's see if I've understood that well enough to put all those pieces together: RequestEncoder.parser(media_type) looks for the Mime Type of the media_type that got passed to it, gets the encoder for that type from a list of encoders (or a default), and then gets the response_parser from that encoder.

That leaves us with, where the heck is media_type getting defined?

Github says its defined in two places, and one of those is our parent class. So... yeah, okay, ultimately this is about making sure that we parse the response's body with a parser appropriate to its type.

Which means it looks like there's really only two differences between an ActionDispatch::TestResponse and an ActionDispatch::Response

  1. You can create a TestResponse from a Response.
  2. A TestResponse can return its own parsed_body.

A TestResponse can also return the parser it uses.

I'm mildly surprised that Response doesn't have that second capability. Some git digging reveals that parsed_body was added specifically so you don't have to work out how the response was encoded in tests before writing e.g. `JSON.parse(response.body)` I guess... ActionDispatch::Response doesn't get used for Response objects being created by outbound calls, only for preparing inbound calls? I don't really know how or if Rails has a standard way of handling outgoing calls, come to think of it.

Unfortunately, now that I know what ActionDispatch::Response does, it sounds like this isn't the key to the problem that got me interested in this, which was, "What, if anything, should I use instead of assert_select in my RSpec Request tests? Why doesn't has_tag work?"

It does, however, make it clearer that the problem is that has_tag doesn't exist! Careful inspection of the documentation that pointed me in that direction last week reveals that all the references to has_tag are nearly a decade old.

I'll need to do at least a little archaeology to try to figure out why has_tag went away, because a nicely named wrapper around assert_select seems like exactly what I want. Is there a reason that it's not? Should I take another approach (like capybara tests), use a different library, or write my own has_tag method? Join me next week to find out.