NI Grad Jobs


NIGradJobs is a service that scrapes popular job websites in Northern Ireland, upon finding new postings it will post them onto a twitter feed with a link to the job. This is then cross-posted to the Facebook page using IFTTT.

Development

Originally programmed in ES5, the project has now been changed to ES6. The build process includes a babel transpilation to ES5. This was done for readability purposes and access to syntactic sugar such as event => {}.

Features

Twitter API

Uses the twitter API to post up new job findings, the backend for this project uses a simple MongoDB database to store the unique ID for each posting to ensure repeats aren't tweeted.

Configurable

The application uses an object based configuration, allowing for custom operations on scraped data to get required information. For example, NIJobFinder does not use unique IDs in their source code, but includes it in the URL to the posting. Using custom transforms we can get this unique ID directly from the URL using string matching and a regular expression.

{
  url: 'http://www.nijobfinder.co.uk/search/262330730/Page1/',
  rootSelector: '.result',
  selectors: {
    title: '.job-title a',
    image: '.recruiter-logo@src',
    location: '.job-title a@href'
  },
  pagination: '.next a@href',
  limit: 10,
  customTransforms: [function (url) {
    var id = url.match(/\/\d{6}\//g)[0].replace('/', '');
    id = id.replace('/', '');
    return {
      property: 'id',
      value: id
    };
  }]
}