Node.js Built-in Fetch Doesn't Use HTTPS_PROXY: The Second Pitfall When Connecting to Google API

1. Problem Description

In the previous article 'GA4 Data API Proxy Environment Timeout DEADLINE_EXCEEDED', I fixed the gRPC proxy. Following the same environment variable approach, I wrote a data fetching script for Google Search Console—GSC is a regular REST API, so using the built-in fetch is sufficient:

const res = await fetch(
  "https://searchconsole.googleapis.com/webmasters/v3/sites",
  { headers: { Authorization: `Bearer ${token}` } }
);

The performance when running:

TypeError: fetch failed
  cause: ConnectTimeoutError: Connect Timeout Error

In the same terminal:

• curl -x http://127.0.0.1:7897 https://searchconsole.googleapis.com works normally
• HTTPS_PROXY is exported in both uppercase and lowercase
• The grpc_proxy fixed for GA4 in the previous article is still there

Environment variables are all set, but fetch doesn't read any of them.

2. Environment

Item	Details
OS	macOS (Darwin 23.6)
Node.js	22.18.0 (built-in fetch provided by undici)
Target API	Google Search Console (REST)
Proxy	Local Clash mixed port 127.0.0.1:7897

3. Root Cause: This Is Not a Bug, It's a Design

The built-in fetch in Node.js from v18 onwards is based on undici. The default behavior of undici is: completely ignore environment variables like HTTP_PROXY / HTTPS_PROXY / NO_PROXY, and if you need a proxy, you must explicitly configure the dispatcher in the code.

This is contrary to most people's intuition, because almost all common tools read these variables:

Tool / Library	Reads Environment Proxy Variables?
curl / wget	✅ Yes
axios	✅ Yes
gaxios (HTTP layer of the googleapis ecosystem)	✅ Reads uppercase HTTPS_PROXY and lowercase https_proxy
request / node-fetch (agent libraries)	Generally supported by ecosystem solutions
Node built-in fetch (undici)	❌ No by default

The typical pitfall path is: first use curl to verify that the proxy is fine, then write the fetch version, and then question your life over the timeout—the success of curl is precisely the misleading factor, as it and fetch are not in the same proxy discovery mechanism.

4. Three Fix Paths

Path 1 (Recommended): Don't Write Fetch Barely for Google API, Use google-auth-library's client.request()

If your scenario is similar to mine, calling Google APIs, the smoothest solution is to let the authentication library handle the HTTP layer as well:

import { GoogleAuth } from "google-auth-library";

const auth = new GoogleAuth({
  scopes: ["https://www.googleapis.com/auth/webmasters.readonly"],
});
const client = await auth.getClient();

// client.request uses gaxios under the hood, which honors HTTPS_PROXY
const res = await client.request({
  url: "https://searchconsole.googleapis.com/webmasters/v3/sites",
});

This solution has two additional benefits:

• Token refresh is automatically handled, no need to manually construct the Authorization header
• If you are already using official SDKs like @google-analytics/data, google-auth-library is their transitive dependency, no additional installation is needed, and it also avoids the large googleapis package

gaxios has the same strict requirements for proxy addresses as grpc-js—the value must have the http:// prefix; writing 127.0.0.1:7897 directly will throw Invalid URL.

Path 2 (General): undici ProxyAgent

For scenarios where you want to use fetch but not with Google APIs, explicitly configure the dispatcher for undici:

import { ProxyAgent, setGlobalDispatcher } from "undici";

setGlobalDispatcher(new ProxyAgent("http://127.0.0.1:7897"));

// every fetch in this process now goes through the proxy
const res = await fetch("https://example.com");

If you don't want to affect globally, pass it per request:

const res = await fetch(url, {
  dispatcher: new ProxyAgent("http://127.0.0.1:7897"),
});

(dispatcher is an undici extended option, and type assertions may be needed in TypeScript.)

Path 3 (Newer Node): Experimental Environment Variable Switch

Newer versions of Node.js provide an experimental NODE_USE_ENV_PROXY=1 to make the built-in fetch respect environment proxy variables. If your Node version supports this and you can accept experimental features, this is the solution with minimal changes—but in projects locked to Node 22 (like mine), the first two paths are more stable. Test it on the target version before use, and don't write experimental switches into production scripts.

5. Combined with the Previous Article: Quick Reference for Google API Proxy Configuration

With the same Google credentials, the transport layers of different APIs are completely different, so the proxy configurations are also completely different:

API	Transport Layer	Proxy Configuration
GA4 Data API (@google-analytics/data)	gRPC	Lowercase grpc_proxy=http://... (uppercase HTTPS_PROXY is ineffective)
Search Console / most googleapis	REST (gaxios)	HTTPS_PROXY=http://... (recognized in both cases)
Writing fetch yourself	undici	All environment variables are ineffective; must configure ProxyAgent in code

My actual approach is to uniformly prefix the script with two variables to cover the first two cases:

HTTPS_PROXY=http://127.0.0.1:7897 grpc_proxy=http://127.0.0.1:7897 node script.js

6. Troubleshooting Mantras

• curl works ≠ your code will work: curl has its own proxy discovery mechanism, which cannot verify the behavior of the HTTP library you use
• When changing libraries, first ask: Does this library read environment proxy variables? Which one? What case?—directly browse the source code in node_modules; it's faster than checking documentation and won't be outdated
• When Node built-in fetch times out and the environment has a proxy, the first reaction should be that undici doesn't read environment variables; don't waste time checking DNS and firewalls