Scheduling jobs

The DataCleaner monitor web application is the preferred application for scheduling of DataCleaner jobs.

Below is a screenshot of the Scheduling page of the DataCleaner monitor. To access this page, your user needs to have the ROLE_SCHEDULE_EDITOR security role.

Things to notice on this page:

  1. Each job is categorized by their datastores. This is to provide an overview of which jobs are executing based on which datastores. Technically, a job can interact with more than one datastore, but it will be categorized on this page based on which datastore is acting as source for the job's execution flow.

  2. Besides the datastore and job name, we have a Scheduling column. In the case shown, all jobs are 'Manually triggered', meaning that they are not scheduled by the DataCleaner monitor. Click the link to change the trigger:

    Three types of triggering are explained on the screen: Periodic trigger, Dependent trigger, Manual trigger.

    The Hot Folder trigger can be used so that a change to a file (or a folder) can trigger the job to run. If the hot folder (or the folder that contains the configured trigger file) contains a properties file with the same name as the job, or if the configured trigger file is a properties file, its properties are used to override configuration elements of DataCleaner monitor when executing the job.

  3. In the next column Alerts are defined. Alerts define conditions that should be inspected after execution. An alert is based on metrics, similar to those that can be shown in the timelines on the Dashboard page. An alert also contains a severity, defining how severe the alert is, if it is raised. The severity scale is (increasing severity): Intelligence, Surveillance, Warning, Fatal .

    Typical handling of raised alerts include sending an email or SMS to the data stewards. These alert notification mechanisms are included in the commercial editions of DataCleaner. You can also develop your own alert notification handlers, by implementing the org.datacleaner.monitor.alertnotification.AlertNotifier interface in Java.

  4. In the last column, Actions can be found. The following options are there:

    1. Trigger , which triggers a job for immediate execution.

    2. Open , which launches the DataCleaner desktop application with the job loaded. Note that for security and bandwidth concerns, only the first 20 input records of the job is transferred to the desktop application. It is in other words not possible to "trigger" the job run on the desktop as if it is running on the server, but it provides a good playground for experiments and testing. Jobs can even be modified and saved back onto the monitoring server this way.

    3. History , which displays a dialog with log information about each triggered job execution.