> ## Documentation Index
> Fetch the complete documentation index at: https://developer.onetrust.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom Scan using Worker Node APIs

The worker node services APIs target Data Discovery worker nodes. These endpoints interact directly with the worker node and allow access to service integrations hosted by the worker nodes. Use the [Custom Scan APIs](/onetrust/reference/geteligiblejobsusingget) to retrieve custom scan jobs, submit metadata for cataloging, and submit content for classification. This guide will walk through all of the required steps to use the [Custom Scan APIs](/onetrust/reference/geteligiblejobsusingget).

## Requirements

* Configure Ingress for Worker Node: for more information, see [API Access for Worker Node Management](https://my.onetrust.com/s/article/UUID-074fbd5c-206a-2efa-7444-b2698da6b4f2).
* Client Credentials: For more information on creating OAuth 2.0 Client Credentials, see [Managing OAuth 2.0 Client Credentials](https://my.onetrust.com/s/article/UUID-3faa8a9f-7635-bcda-5184-c01a157c3132?topicId=0TO1Q000000ItSxWAK).
  * Generate a credential for the custom scanner. Download the credential file containing the `client_id` and `secret`. These values will be used for authentication and authorization for the custom scanner to prove its legitimacy.
* Custom Connector: (Worker Node Scan) Data Source
  * Give your data source a name, set up your credentials, and choose the default settings or opt for an advanced configuration. Once configured, activate the data source. Record the Data Source ID for the  custom connector, which can be obtained from the URL (e.g., <https://{hostname}/data-discovery/data-sources/wizard-details/v2/{Data_Source_ID}/summary>) in the OneTrust application or using the corresponding`id` value from the [Get Data Sources API](/onetrust/reference/getallusingget_1).

## Custom Scan Workflows: How to Steps

1. Use the [Generate Access Token API](/onetrust/reference/getoauthtoken) to retrieve an access token.

2. Use the [Get List of Scan Jobs API](/onetrust/reference/geteligiblejobsusingget) to query the worker node for pending scan jobs for a given data source. Make a note of the Job Identifier in the response body. The `jobId` will be used in the following steps.
   1. `datasourceId` is the Custom Connector: (Worker Node Scan) Data Source identifier from the requirements above. This value can be obtained using the [Get Data Sources API](/onetrust/reference/getallusingget_1).
   2. Optionally, extract the client credential reference from the `credentialReference` response parameter.

3. Once a scan job is available, use the [Update Scan Job Status API](/onetrust/reference/updatejobstatususingput) to update the scan job's status from `PENDING` to `IN_PROGRESS`.

   ```json Update Status to In Progress
   {
     "status": "IN_PROGRESS"
   }
   ```

4. Use the [Submit Data to Catalog API](/onetrust/reference/catalogdatausingpost) to submit catalog metadata for entities in your target data source.

   ```json Database
   {
     "metadataRequestList": [
       {
         "datasourceId": "",
         "jobId": 0,
         "parentEntityName": "Teradata-DS",
         "parentType": "DATASOURCE",
         "parentXPath": [],
         "metadataList": [
           {
             "entityName": "Company",
             "entityType": "DATABASE",
             "childrenCount": 1,
             "description": "Entity Description.",
             "owner": "OT",
             "productVersion": "1",
             "productName": "SQL DB",
             "productFamily": "MySQL"
           }
         ]
       }   
     ]
   }
   ```
   ```json Schema
   {
     "metadataRequestList": [
     {
         "datasourceId": "",
         "jobId": 0,
         "parentEntityName": "Company",
         "parentType": "DATABASE",
          "parentXPath": [
          "Company"
         ],
         "metadataList": [
           {
             "entityName": "TEST-SCHEMA",
             "entityType": "SCHEMA",
             "childrenCount": 1,
             "description": "Entity Description.",
             "version": 1
           }
         ]
       }
     ]
   }
   ```
   ```json Object
   {
     "metadataRequestList": [
       {
         "datasourceId": "",
         "jobId": 0,
         "parentEntityName": "TEST-SCHEMA",
         "parentType": "SCHEMA",
         "parentXPath": [
           "Company",
           "TEST-SCHEMA"
         ],
         "metadataList": [
           {
             "entityName": "Employee",
             "entityType": "OBJECT",
             "childrenCount": 3,
             "description": "Entity Description.",
             "rowsEstimate": 1000,
             "sizeEstimate": 120,
             "estimateTimestamp": 12345678,
             "rowsCalculated": 1000,
             "sizeCalculated": 120,
             "calculatedTimestamp": 12345678,
             "comments": "sample_comment_about_employee_table"
           }
         ]
       }
     ]
   }
   ```
   ```json Table
   {
     "metadataRequestList": [
       {
         "datasourceId": "",
         "jobId": 0,
         "parentEntityName": "TEST-SCHEMA",
         "parentType": "SCHEMA",
         "parentXPath": [
           "Company",
           "TEST-SCHEMA"
         ],
         "metadataList": [
           {
             "entityName": "Employee",
             "entityType": "TABLE",
             "childrenCount": 3,
             "description": "Entity Description.",
             "rowsEstimate": 1000,
             "sizeEstimate": 120,
             "estimateTimestamp": 12345678,
             "rowsCalculated": 1000,
             "sizeCalculated": 120,
             "calculatedTimestamp": 12345678,
             "comments": "sample_comment_about_employee_table"
           }
         ]
       }
     ]
   }
   ```
   ```json Field
   {
     "metadataRequestList": [
       {
         "datasourceId": "",
         "jobId": 0,
         "parentEntityName": "Employee",
         "parentType": "OBJECT",
         "parentXPath": [
           "Company",
           "TEST-SCHEMA",
           "Employee"
         ],
         "metadataList": [
           {
             "entityName": "first_name",
             "entityType": "FIELD",
             "description": "Entity Description.",
             "columnIndex": 1,
             "nullable": true,
             "columnDescription": "contains first_name",
             "encryption": "AES",
             "sizeEstimate": 10,
             "averageSizeCalculated": 15,
             "calculatedTimestamp": 12345
           }
         ]
      }
     ]
   }
   ```
   ```json Column
   {
     "metadataRequestList": [
       {
         "datasourceId": "",
         "jobId": 0,
         "parentEntityName": "Employee",
         "parentType": "TABLE",
         "parentXPath": [
           "Company",
           "TEST-SCHEMA",
           "Employee"
         ],
         "metadataList": [
           {
             "entityName": "first_name",
             "entityType": "COLUMN",
             "description": "Entity Description.",
             "columnIndex": 1,
             "nullable": true,
             "columnDescription": "contains first_name",
             "encryption": "AES",
             "sizeEstimate": 10,
             "averageSizeCalculated": 15,
             "calculatedTimestamp": 12345
           },
           {
             "entityName": "last_name",
             "entityType": "COLUMN",
             "description": "Entity Description.",
             "columnIndex": 2,
             "nullable": true,
             "columnDescription": "contains last_name",
             "encryption": "AES",
             "sizeEstimate": 10,
             "averageSizeCalculated": 15,
             "calculatedTimestamp": 12345
           },
           {
             "entityName": "email",
             "entityType": "COLUMN",
             "description": "Entity Description.",
             "columnIndex": 3,
             "nullable": true,
             "columnDescription": "contains email",
             "encryption": "AES",
             "sizeEstimate": 10,
             "averageSizeCalculated": 15,
             "calculatedTimestamp": 12345
           }
         ]
      }
     ]
   }
   ```
   ```json Folder
   {
     "metadataRequestList": [
   	{
         "datasourceId": "",
         "jobId": 0,
         "parentEntityName": "SMB-DS",
         "parentType": "DATASOURCE",
         "parentXPath": [

         ],
         "metadataList": [
           {
             "entityName": "TEST-FOLDER",
             "entityType": "FOLDER",
             "childrenCount": 1,
             "description": "Entity Description.",
   		  "sizeOnDisk": 372,
             "dataTotalSize": 0,
             "createdAt": 0,
   		  "lastModified": 0,
   		  "modifiedByUser": "user",
   		  "createdByUser": "user",
   		  "accessedTimestamp": 0,
   		  "fileOwner": "user"
   		  "encryption": "",
   		  "sharedAccess": "SHARED_EXTERNAL",
   		  "url": "",
   		  "eTag": ""
           }
         ]
       }
     ]
   }
   ```
   ```json File
   {
     "metadataRequestList": [
   	{
         "datasourceId": "",
         "jobId": 0,
         "parentEntityName": "TEST-FOLDER",
         "parentType": "FOLDER",
         "parentXPath": [

         ],
         "metadataList": [
           {
             "entityName": "TEST-FILE",
             "entityType": "FILE",
             "childrenCount": 1,
             "description": "Entity Description.",
   		  "sizeOnDisk": 372,
             "dataTotalSize": 0,
             "createdAt": 0,
   		  "lastModified": 0,
   		  "modifiedByUser": "user",
   		  "createdByUser": "user",
   		  "accessedTimestamp": 0,
   		  "fileOwner": "user"
   		  "encryption": "",
   		  "sharedAccess": "SHARED_EXTERNAL",
   		  "url": "",
   		  "eTag": ""
           }
         ]
       }
     ]
   }
   ```

5. For entities with content, use the [Submit Data to Classify API](/onetrust/reference/classifydatausingpost) to submit the content for classification.
   ```json Classification Request
   {
     "jobId": 10,
     "datasourceId": "",
     "totalCount" : 1000,
     "content": [
       {
         "entityName": "first_name",
         "entityType": "COLUMN",
         "parentEntityName": "Employee",
         "parentEntityType": "TABLE",
         "parentXpath": [
           "Company",
           "Employee"
         ],
         "data": [
           "Donald"
         ], 
         "endOfContent": true
       },
       {
         "entityName": "last_name",
         "entityType": "COLUMN",
         "parentEntityName": "Employee",
         "parentEntityType": "TABLE",
         "parentXpath": [
           "Company",
           "Employee"
         ],
         "data": [
           "Trump"
         ],
         "endOfContent": true,
         "totalCount" : 1000
       },
       {
         "entityName": "email",
         "entityType": "COLUMN",
         "parentEntityName": "Employee",
         "parentEntityType": "TABLE",
         "parentXpath": [
           "Company",
           "Employee"
         ],
         "data": [
           "jack@ot.com"
         ],
         "endOfContent": true,
         "totalCount" : 1000
       }
     ]
   }
   ```

6. When all entities have been extracted, use the [Update Scan Job Status API](/onetrust/reference/updatejobstatususingput) to update the scan job's status from `IN_PROGRESS` to `COMPLETED`.
   ```json Update Status to Completed
   {
     "status": "COMPLETED"
   }
   ```