JS Reverse Engineering Case Study #4 — A Dynamic Request-Header Signature (HMAC + Custom Character Table + `execjs`)

Part 4 of the "JS Reverse Engineering in Practice — 30 Case Studies" series.
Source code (this case): JS Reverse Engineering — 30 Case Studies · GitHub (Project4)

Note on sanitization: This is a sanitized, educational write-up. Every real domain, endpoint, site-specific constant and cookie name has been replaced with a generic placeholder (example.com, <...>). The custom character table has been reduced to placeholders. No production endpoint or secret is disclosed. The purpose is to explain the technique, not to enable abuse of any specific service.

1. Goal & Background

The target is the company-search API of an enterprise business-information lookup platform (referred to as "Site D"). Its most striking feature is that the signature does not have a fixed name: both the header key and the header value are generated dynamically and may differ on every request, with x-pid being the only thing that ties the two together.

Target site: https://www.example.com/
Target endpoint: https://www.example.com/<search-endpoint>
Parameters to reconstruct: one dynamic request header (neither the header key nor its value is fixed) plus x-pid
Difficulty: ⭐⭐⭐⭐ — the algorithm is HMAC, but the key is derived from "the path run through a custom character table" and the value is salted with the page's window.tid, making the chain longer than the previous case.

This project is the archetype of the dynamic request header: the server does not recognise "some fixed header name"; it sends down a pair — a header name that is 20 characters from the middle of one hash, and a value that is a separate HMAC — and validates them together, with x-pid binding the browser session. So the reverse-engineering effort has two jobs: how the header name/value are computed, and where tid / pid come from.

2. Traffic Analysis

Open https://www.example.com/, search for any keyword (say <keyword>), press F12 → Network → Fetch/XHR, and find the search POST.

Look at the Request Headers and you will notice something counter-intuitive — besides the normal headers, there is a header name you have never seen before, and it may change to another name after every refresh:

accept / origin / referer / user-agent ... (normal headers) x-pid: web_<SESSION_ID> <20-char lowercase header name>: <a long hex value>

Now look at the Payload:

{"searchKey":"<keyword>","pageIndex":2,"pageSize":20}

The request body is plaintext. The x-pid header is an ID with a fixed prefix, and combined with that "random header name" I concluded:

x-pid corresponds to the page's window.pid (a session-level identifier);
the "random header name + value" is a signature pair: the name is derived from the endpoint path + the request body, and the value additionally mixes in a window.tid.

📷 [Figure 1 — screenshot suggestion] In the Network panel: the dynamic request header and x-pid of the search request.

Conclusion: what must be reconstructed is the algorithm for the "header name / header value", plus extracting window.pid / window.tid from the page.

3. Locating the Signature Entry Point

First, in the Sources panel, globally search for search to reach the request wrapper; then search for the point where the header is assigned, and you will see it call something like getXXX(...) to generate that dynamic header.

Following it further in, I found the core signing module. I carved this logic out into js/arg.js in the repo. Its structure is: one hash implementation (HMAC-SHA256) + one "path → key" character-table mapping function + two assembly functions:

// From the real repo JS: Project4/<site>/js/arg.js (core module, excerpt) AA = { // HMAC over e with key t, returned as a string default:function(e, t) { return (0, AA_3.default)(e, t).toString() } } AA_2 = { "default": { "n": 20, "codes": { // a fixed 20-entry table (digits 0-19 -> letters) "0": "<c0>", "1": "<c1>", "2": "<c2>", "3": "<c3>", "4": "<c4>", "5": "<c5>", "6": "<c6>", "7": "<c7>", "8": "<c8>", "9": "<c9>", "10": "<c10>", "11": "<c11>", "12": "<c12>", "13": "<c13>", "14": "<c14>", "15": "<c15>", "16": "<c16>", "17": "<c17>", "18": "<c18>", "19": "<c19>" } } } AA_3 = { // internally PP.HMAC(SHA-256) default:function(e, n) { return new PP.HMAC.init(TT,n).finalize(e) } } OO = { default:function() { for (var e = (arguments.length > 0 && void 0 !== arguments[0] ? arguments[0] : "/").toLowerCase(), t = e + e, n = "", i = 0; i < t.length; ++i) { var o = t[i].charCodeAt() % AA_2.default.n; n += AA_2.default.codes[o] } return n } } RR = { default:function() { var e = arguments.length > 1 && void 0 !== arguments[1] ? arguments[1] : {} , t = arguments.length > 2 && void 0 !== arguments[2] ? arguments[2] : "" , n = (arguments.length > 0 && void 0 !== arguments[0] ? arguments[0] : "/").toLowerCase() , i = JSON.stringify(e).toLowerCase(); return (0, AA.default)(n + "pathString" + i + t, (0, OO.default)(n)) } }

Once the three functions are clear, everything falls into place:

AA.default(e, t): runs HMAC over the string e with key t (AA_3 is internally PP.HMAC.init(TT, n).finalize(e), using a SHA-256 implementation — you can see the SHA-256 initial constant 1779033703 and the round constant 1116352408 in the file);
OO.default(path): lowercases path, duplicates it (t = e + e), takes charCodeAt() % 20 for each character, looks the letter up in the codes table, and concatenates a fixed-length key. This "character-table mapping" is what makes the key look random without the server having to send it down;
RR.default(path, data, tid): HMAC(path + "pathString" + JSON(body).toLowerCase() + tid, OO(path)).

Finally there is the entry point get_arg exported for Python, which returns the "header name" and "header value" together:

// From the real repo JS: Project4/<site>/js/arg.js (tail) TT = "/<search-endpoint>" EE = { data:{ "searchKey": name, "pageIndex": page, "pageSize": 20 } } SS = { default:function() { var list = ["w", "i", "n", "d", "o", "w", ".", "t", "i", "d"]; return eval(list.join("")) } } oX = { // generates the "header name" default:function() { var a = AA, o = OO var e = arguments.length > 1 && void 0 !== arguments[1] ? arguments[1] : {} , t = (arguments.length > 0 && void 0 !== arguments[0] ? arguments[0] : "/").toLowerCase() , n = JSON.stringify(e).toLowerCase(); return (0, a.default)(t + n, (0, o.default)(t)).toLowerCase().substr(8, 20) } } u = (0, RR.default)(TT, EE.data, (0, SS.default)()); var i = (0, oX.default)(TT, EE.data) res = { key:i, value:u } return res; }

The line eval(list.join("")) in SS.default simply spells out window.tid (the author deliberately split it into an array to defeat static search), i.e. the first argument passed into get_arg(WindowTid, ...).

So:

header name key = HMAC(path + JSON(body).toLowerCase(), OO(path)).toLowerCase().substr(8, 20) → characters 8-27 of the hash result, i.e. 20 lowercase characters;
header value value = HMAC(path + "pathString" + JSON(body).toLowerCase() + window.tid, OO(path)).

The signature of get_arg(WindowTid, name, page) is function get_arg(WindowTid,name,page){ window = global; window.tid = WindowTid; ... }: it first sets window = global and window.tid = WindowTid to patch the environment, so that the later SS.default can obtain window.tid via eval.

📷 [Figure 2 — screenshot suggestion] In the Sources panel: a breakpoint at the oX / RR call site, or the HMAC call stack.

4. Restoring the JS & Emulating the Environment

The only environment patching in this file is a single window = global; almost everything else is pure logic. The real trouble is the prototype-chain issue when "carving the code out" — the author left a very detailed "fix summary" comment at the end of the file, recording five pitfalls:

// From the real repo JS: Project4/<site>/js/arg.js (file-tail fix comment, excerpt) /* Fix summary (5 places): create.toString / AA_4.init.prototype.toString / tox32's i.create; plus _doProcessBlock using `for (var t=0;t<80;t++)`, where `var` shadows the parameter t (the data array); and mishandling extend and e.init, which permanently rewrites cfg.init, making the hash object's high/low both NaN. Restored to the original e.$super.init.apply(this, arguments). */

What these pitfalls have in common is: when hand-carving crypto-js / a home-made hash, the variable scope and prototype chain of extend, init and _doProcessBlock are so sensitive that the slightest slip makes the hash come out wrong (and it usually does not throw — it just returns the wrong result). Also, in OO.default the character table's n=20 and the codes mapping are hard-coded, and not a single character may be changed when restoring them; path is .toLowerCase()-processed before and after computation, so passing an uppercase path yields a different header name.

On the Python side, all you need to supply is WindowTid (extracted from the page); everything else is computed inside the JS.

5. Python Implementation

First, the "extract pid / tid from the page" part, py/arg/<site>_arg.py: request the search page and parse the inline <script>window.pid='...' with string splitting:

# From the real repo py: Project4/<site>/py/arg/<site>_arg.py import requests def site_arg(name:str)-> dict[str,str]: headers:dict[str,str] = { # ... ordinary browser headers (accept / accept-language / cache-control / dnt /  # pragma / sec-ch-ua* / sec-fetch-* / user-agent etc.), trimmed here  "referer": "https://www.example.com/", "priority": "u=0, i", "upgrade-insecure-requests": "1", "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36" } cookies:dict[str,str] = { "<DEVICE_COOKIE>": "<your-device-cookie>", "<WAF_COOKIE>": "<your-waf-cookie>", "<SESSION_COOKIE>": "<your-session-cookie>", "<ANTIBOT_COOKIE>": "<your-antibot-cookie>" } url:str = "https://www.example.com/web/search" params:dict[str,str] = { "key": name, "p": "2" } response:requests.Response = requests.get(url, headers=headers, cookies=cookies, params=params) result:str = response.text WindowPid:str = result.split("<script>window.pid='")[1].split("'")[0] WindowTid:str = result.split("<script>window.pid='")[1].split("='")[1].split("';")[0] WindowVersion:str = result.split("'</script>")[0].split("window.version='")[1] return {"WindowPid":WindowPid, "WindowTid":WindowTid, "WindowVersion":WindowVersion}

Then the main script py/Spider_<site>.py: fetch WindowPid / WindowTid, use execjs to compute the headers, and splice them into the request:

# From the real repo py: Project4/<site>/py/Spider_<site>.py import requests import json from arg.site_arg import site_arg import os import execjs from urllib.parse import quote from typing import Union,Any def Spider(Name:str,Page:int)-> None: name:str = Name pageIn:int = Page header_refererNa:str = quote(name,encoding="utf-8") header_referer:str = "https://www.example.com/web/search?key=" + header_refererNa + "&p=2" if pageIn == 1 else "https://www.example.com/web/search?key=" + header_refererNa arg:dict = site_arg(name) WindowPid:str = arg["WindowPid"] WindowTid:str = arg["WindowTid"] base_path:str = os.path.dirname(os.path.abspath(__file__)) js_path:str = os.path.join(base_path,"..","js","arg.js") js_dir:str = os.path.dirname(js_path) os.chdir(js_dir) with open(js_path,encoding="utf-8") as f: js_code:str = f.read() js:Any = execjs.compile(js_code) hea_arg:dict[str,str] = js.call("get_arg",WindowTid,name,pageIn) key:str = hea_arg["key"] value:str = hea_arg["value"] headers:dict[str,str] = { "Content-Type": "application/json", key: value, # <- the dynamic header name : value  # ... ordinary browser headers (accept / origin / referer / sec-ch-ua* / user-agent etc.), trimmed here  "origin": "https://www.example.com", "referer": header_referer, "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36", "x-pid": WindowPid, "x-requested-with": "XMLHttpRequest" } cookie:dict[str,str] = {"<DEVICE_COOKIE>":"<your-device-cookie>", "<WAF_COOKIE>":"<your-waf-cookie>", "<SESSION_COOKIE>":"<your-session-cookie>"} url:str = "https://www.example.com/<search-endpoint>" Data:dict = {"searchKey": Name, "pageIndex": pageIn, "pageSize": 20} data:str = json.dumps(Data, separators=(',', ':')) response:requests.Response = requests.post(url, headers=headers, cookies=cookie, data=data) print(response.text) print(response.status_code) Spider("<keyword>",2)

Key points:

The dynamic header is { key: value } inserted directly into headers (key is that 20-character random name, value is the HMAC result);
x-pid uses WindowPid, and referer uses the URL-encoded search page address;
pageIndex is passed the Page value (here it is one-based, unlike the previous case);
The cookies (<DEVICE_COOKIE> / <WAF_COOKIE> / <SESSION_COOKIE>) are login state, already sanitized to placeholders, and must be replaced.