One bug, one month of debugging

I wanted to share about a minor bug that took me 1 month to resolve. The reason for that was, my lack of knowledge regarding HTTP. I believe my mistake is noteworthy and common, especially among new web developers jumping straight to frameworks instead of the basics.

# Background

The project is a combination of,

• Remix website running on Vercel
• An Express server & a database running on a DigitalOcean Droplet

The website supports 2 languages, and I had to store each user’s language preference somewhere. I was planning to use local storage, however, I changed my mind to try out HTTP cookies. The browser stores cookies locally, and includes them in future requests to the same domain.

When a user visits the website, the request is sent to loaders, running on Vercel’s Edge Runtime. To load data from the database, the Express server is then fetched from the loaders.

While the loaders received cookies (from browsers), the Express server did not receive the cookies. To workaround this issue, I made the following change.

export const loader = async ({ request }) => {
	const cookieHeader = request.headers.get("Cookie");
	const response = await fetch(__SERVER_URL__, {
		headers: {
			Cookie: cookieHeader,
		},
	});
	// ... continue
};

In case the code wasn’t clear, we are adding the Cookie header manually, when fetching the Express server. We are forwarding the Cookie header. I honestly thought it’s straightforward and simple. Moreover, it worked when I tested it locally. But it didn’t work on production.

I had no idea why. I was adding console.logs here and there, fiddling with server configuration, fiddling with cookie options and doing everything. The server logs were not helpful either.

It took me about a month to find the cause.

# The Cause

While reading through MDN, I found the following paragraph:

I went to the forbidden header names page, and found Cookie listed there. And I finally found the issue. User agent can only control the Cookie header (for obvious security reasons). Take a look at the table below.

When testing locally, the loaders were running on Node.js. The Cookie Forwarding worked as Node.js doesn’t restrict us from using Cookie header. However, when running on Vercel, Vercel Edge Runtime was the user agent, and it did not allow setting Cookie header.

I was finally at peace. 😮‍💨

# The Solution

The solution was straight forward. The user agent controls the forbidden headers only. We can use a non-forbidden header, or a custom one. And I switched to using a custom header.

export const loader = async ({ request }) => {
	const cookieHeader = request.headers.get("Cookie");
	const response = await fetch(__SERVER_URL__, {
		headers: {
			"Forwarded-Cookie": cookieHeader,
		},
	});
	// ... continue
};

Now, the request to Express server will include the required cookies. Note that we have to change the cookie parsing logic in the server to make this work. To fix this, I ditched cookie-parser package, and wrote a custom middleware that supports new Forwarded-Cookie header. The new middleware used the same cookie package which cookie-parser uses.