An IT Jobs Filter Can Exclude a Bank's Engineers

Your IT-jobs corpus can lose a bank's engineers before the embedding model sees a job description.

You set industries to a technology sector, expecting software roles. The collection now answers a different question: which jobs belong to technology employers? That's an uncomfortable discovery after you've started tuning retrieval.

Lab note 059: which fields should an occupation-based collection preserve so the employer's business doesn't become a proxy for the work?

The catch: this is a schema-design verdict, not measured retrieval quality. A taxonomy label also isn't proof of someone's skills.

Which jobs belong in your IT corpus?

Suppose you're building retrieval for software engineering vacancies across employers. A bank's engineering role belongs; an accounting role at a technology employer doesn't qualify merely because of its employer.

Our hypothesis: preserve the occupational path and employer sector separately, then use the description as evidence of the actual work.

Our jobs.ch Actor exposes that distinction through categories and industries.

Example occupational browsing input, with valid category IDs illustrating the request shape rather than an asserted IT mapping:

{ "mode": "browse", "categories": ["175", "176"], "includeDetails": true, "descriptionFormat": "text" } 

Example employer-sector browsing input, with a valid industry ID illustrating a sector selection:

{ "mode": "browse", "industries": ["1"], "includeDetails": true, "descriptionFormat": "text" } 

These requests define different populations. Check the identifier labels against your collection's scope before using either selection in production.

That choice comes before chunk sizes and embedding models. Those operate on the population you admit.

The intersection happens before embeddings

industry describes the employer's business sector; category describes the job's occupational function.

Multiple values within a filter use OR. Different filters use AND.

Adding an industry constraint to an occupational request therefore creates an intersection. It doesn't provide another interpretation of “tech.”

This hypothetical split shows the eligibility decision:

IT occupation selected | +-- employer sector unrestricted --> bank engineer remains eligible | +-- technology sector required ----> bank engineer excluded 

“Eligible” matters. Passing a predicate doesn't guarantee that a listing exists, carries the expected label, or reaches your bounded collection.

An evaluation set drawn only from collected rows can conceal this mistake. It can tell you whether retrieval finds indexed engineering roles while saying nothing about careers excluded at ingestion.

Keep a synthetic bank-engineer case in your collection checks. Otherwise, ranking gets blamed for a decision made upstream.

Give every field a job

We'd assign fields according to their purpose, rather than embed the entire record and hope the labels explain themselves.

We wouldn't silently coalesce industry and companyIndustry. If their values differ, that's something to inspect. A convenient fallback expression would hide the distinction.

Save collection inputs in your ingestion manifest too. A row's occupational label can't reconstruct which filter combination admitted it.

Keep the evidence, drop the decorations

For this task, logos, application mechanics, contact details and employer ratings don't help establish what work a vacancy involves. Leave them out of embedding text.

That doesn't require deleting them from storage. Another retrieval task or a display layer may need them later.

A title alone is weak evidence of scope. “Engineer” can describe quite different work; responsibilities and requirements give the model something to check.

Employer sector still deserves structured metadata. A later query for engineering roles in banking can apply that predicate deliberately against a broader collection.

IDs and URLs can stay outside embedding text. Citation identity and semantic evidence have different jobs.

For chunking, repeat the labeled role and taxonomy context with each description chunk. Otherwise, a paragraph about deployment or reporting may arrive without enough context to identify the vacancy's function.

Avoid treating an arbitrary description prefix as representative. Requirements, responsibilities and exclusions can appear in different parts of the text. A smaller token bill isn't much comfort if the omitted paragraph carried the answer.

A serializer that keeps the meanings attached

This serializer builds labeled prompt text while retaining the taxonomy values as structured metadata.

import json def field_text(value): if value is None: return "" if isinstance(value, str): return value.strip() if isinstance(value, (list, dict)) and not value: return "" return json.dumps(value, ensure_ascii=False, sort_keys=True) def serialize_job(job): job_id = field_text(job.get("jobId")) if not job_id: raise ValueError("A jobId is required for index identity") labeled_fields = ( ("Role title", "title"), ("Employer", "companyName"), ("Occupation path", "category"), ("Employer sector, listing", "industry"), ("Employer sector, company profile", "companyIndustry"), ) context = [ f"{label}: {field_text(job.get(field)) or 'Not supplied'}" for label, field in labeled_fields ] description = field_text(job.get("description")) or "Not supplied" return { "jobId": job_id, "jobUrl": job.get("jobUrl"), "category": job.get("category"), "industry": job.get("industry"), "companyIndustry": job.get("companyIndustry"), "text": "\n".join(context) + "\n\nJob description:\n" + description, } 

“Not supplied” is our serializer's marker. The Actor omits empty values, so absence should remain visible rather than become an inferred occupational label.

The function preserves structured values as JSON text without guessing at nested keys. Treat the description as source material in your prompt, rather than instructions for the answering model.

Example synthetic records make the distinction inspectable without pretending they're collected vacancies:

example_jobs = [ { "jobId": "example-bank-engineer", "title": "Software engineer", "companyName": "Example Bank", "category": "IT > Software development", "industry": "Banking", "companyIndustry": "Financial services", "description": "Example role maintaining payment-processing software.", "jobUrl": "https://example.com/jobs/bank-engineer", }, { "jobId": "example-technology-accountant", "title": "Accountant", "companyName": "Example Technology Employer", "category": "Finance > Accounting", "industry": "Information technology", "companyIndustry": "Information technology", "description": "Example role preparing accounts and financial reports.", "jobUrl": "https://example.com/jobs/technology-accountant", }, ] example_contexts = [serialize_job(job) for job in example_jobs] 

Inspect example_contexts in your notebook. The labels distinguish the work from the employer's business without asking a model to interpret an unlabeled bag of taxonomy values.

For relevance evaluation, these synthetic cases serve different purposes: the engineering vacancy checks inclusion across sectors, while the accountant checks whether employer identity overwhelms evidence about the role.

They express the intended task contract. They don't establish real-world classification accuracy.

What this can't fix

Switching to semantic search doesn't remove an unintended industry constraint. Its filters still apply.

A richer serializer can't recover excluded documents either. The embedding model has no appeals department.

Category-based collection has its own boundary: a relevant role may lack the label you expected. For broader coverage, consider a separate keyword collection, then reconcile overlapping records on jobId. Evaluate that strategy rather than assuming the taxonomy is exhaustive.

The plain verdict: keep occupation and employer sector as separate axes, and constrain each only when the user's question calls for it.

That resolves the schema ambiguity. Retrieval recall and market coverage still need their own evaluation.

Wire the serializer into collection

The worked example uses Jobs.ch Swiss Job Listings by Region & Industry.

This collection cell applies the serializer and routes records without identity or description text into a review queue.

import os from apify_client import ApifyClient run_input = { "mode": "browse", "categories": ["175", "176"], "includeDetails": True, "descriptionFormat": "text", "maxItems": 20, } client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor( "vexaloom-labs/jobs-ch-swiss-job-listings-by-region-and-industry" ).call(run_input=run_input) if run["status"] != "SUCCEEDED": raise RuntimeError(f"Collection ended with status {run['status']}") items = client.dataset(run["defaultDatasetId"]).list_items().items index_records = {} review_queue = [] for item in items: if not field_text(item.get("jobId")) or not field_text( item.get("description") ): review_queue.append(item) continue prepared = serialize_job(item) index_records[prepared["jobId"]] = prepared 

Within this batch, repeated IDs replace the previous index entry. Across snapshots, use an explicit update policy so an older or less complete record doesn't overwrite richer evidence.

Include the review queue in coverage reporting. Missing description text is an ingestion condition, not evidence that an occupation is irrelevant.

Keep raw records available for rebuilding the index. Changing your serializer should require reprocessing data, not recollecting the web.

Reproduce the filter choice

• Inspect the synthetic contexts for explicit occupational and sector labels.
• Check the illustrative category selections against the labels needed for your IT scope.
• Save the baseline configuration alongside its dataset.
• Compare a sector-constrained variant separately, checking population eligibility before retrieval rankings.

Example exact baseline input, retaining the illustrative selections so you can reproduce the request shape:

{ "mode": "browse", "categories": ["175", "176"], "includeDetails": true, "descriptionFormat": "text", "maxItems": 20 }