Promptwatch Logo

WebSpiderMount

Job wrapping data processor handling jobs distribution from employer websites to multiple endpoints, like job boards, advertisement platforms, job alerts etc.
AspenTechLabs Incwebspidermount
Aggregator

What is WebSpiderMount?

WebSpiderMount moves vacancy data from an employer site or applicant tracking system into the format required by a job board. AspenTechLabs Inc calls this job wrapping. Clients use it to distribute listings without posting each vacancy again by hand.

During a scheduled run, the spider searches the source, opens selected job pages, and extracts fields from their HTML. It can clean or map the data before saving it as XML or CSV, or sending it through a recipient API. Later runs add new vacancies, revise changed records, and remove expired posts from the destination.

The bot record supplies the token webspidermount and a browser-style user agent containing WSM/2.0. AspenTechLabs says employer career sites and applicant tracking systems are its normal sources. Commercial job boards are scraped only when a formal agreement allows it. The operator's public material does not promise robots.txt compliance for this client.

This is employment-data distribution, not AI search or model training. For a participating employer, blocking the spider can leave downstream job boards incomplete or stale. There is no documented connection between these fetches and visibility in AI-generated answers.

Indirectly relevant

Is WebSpiderMount relevant for AI search?

Indirectly. WebSpiderMount has no AI product of its own, but its output can end up in the systems that AI answers draw on.

WebSpiderMount collects content from many sites and redistributes it through its own product. Aggregated copies can end up in datasets that AI systems later learn from, and some aggregators are themselves sources that AI answer engines draw on.

How to handle WebSpiderMount

Organizations using the service should allow the agreed job-search and vacancy URLs. Keep application forms, candidate information, and administrative screens outside the spider's reach through ordinary authorization and narrow path rules.

The stable token supports this robots preference:

User-agent: webspidermount
Disallow: /

AspenTechLabs does not document compliance, so confirm the outcome in logs. When there is no job-wrapping agreement, ask the operator or client to remove the source and apply server-side enforcement if the preference is ignored.

Examples

  • An employer exposes `/careers/jobs/` to WebSpiderMount so its recruitment partner receives listing updates, while candidate applications remain authenticated.
  • A job board with no scraping agreement sees `WSM/2.0` in its logs, confirms that no client authorized the collection, and stops the requests at the server.

Frequently asked questions about WebSpiderMount

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

It distributes vacancy records copied from employer career sites or applicant tracking systems to job boards and other recruitment destinations.

Related bots

AirsoftdbBot

Airsoftdb is a search engine for airsoft prod.

Aggregator

Artsdata Crawler

Web crawler that collects publicly available LOD for arts and culture in Canada.

Aggregator

BestChange Bot

The BestChange bot downloads exchange rate information from 600 websites every 5 seconds.

Aggregator

BoardGamePrices Bot

Price comparison site for board games. Need to crawl store pages for participating stores. All stores give permission to be crawled.

Aggregator

Capital One Bot

Capital One Bot crawls dealer websites for getting the usage information for Capital One lead navigator button.

Aggregator

Be the brand AI recommends

Monitor your brand's visibility across ChatGPT, Claude, Perplexity, and Gemini. Get actionable insights and create content that gets cited by AI search engines.

Promptwatch Dashboard