Agent 1.x (Deprecated) Documentation

Important

If you are using Agent 1.x, F5 Distributed Cloud App Infrastructure Protection (AIP) recommends that you upgrade your Agent to the newest version.

Deploy Agent 1.x via Amazon AMIs

Below are configuration steps for deploying the Distributed Cloud AIP host-based Agent in your Amazon Machine Image (AMI) environment.

Agent 1.x: Amazon Linux 1

Do not run the cloudsight setup command as part of your Amazon Machine Image (AMI) build process. The cloudsight setup command registers the Linux Agent with the Distributed Cloud AIP service. This registration process assigns a unique token to the Agent. If you include the cloudsight setup command as part of your AMI build process, then the same Agent token will be included on every system deployed using that AMI. This means that multiple Agents will report as a single Agent in Distributed Cloud AIP.

To include the Agent in your AMI build process without registering it:

  1. Install the Distributed Cloud AIP Linux 1.x Series Agent using the Amazon Linux 1 instructions, but do not run the cloudsight setup command.
  2. Run the following command to ensure the Agent does not attempt to start when you boot your instance:
    sudo chkconfig cloudsight off
  3. Make any other necessary configuration changes to your AMI.
  4. Save your AMI.
  5. When you deploy the AMI, as part of your node provisioning or as part of the Amazon User Data script run the following commands:
    sudo cloudsight setup --deploy-key=<your deploy key> --ruleset="Base Rule Set" --agent_type=i
    sudo chkconfig cloudsight on
    sudo service cloudsight start

    Replace <your deploy key> with your Distributed Cloud AIP Linux 1.x series Agent deploy key. The Agent now registers and starts when your client boots.

Agent 1.x: Ubuntu

Do not run the cloudsight setup command as part of your Amazon Machine Image (AMI) build process. The cloudsight setup command registers the Linux Agent with the Distributed Cloud AIP service. This registration process assigns a unique token to the Agent. If you include the cloudsight setup command as part of your AMI build process, then the same Agent token will be included on every system deployed using that AMI. This means that multiple Agents will report as a single Agent in Distributed Cloud AIP.

To include the Agent in your AMI build process without registering it:

  1. Install the Distributed Cloud AIP Linux 1.x Series Agent using the appropriate instructions for your operating system (OS), but do not run the cloudsight setup command.
  2. Run the following command to ensure the Agent does not attempt to start when you boot your instance:
    sudo update-rc.d cloudsight disable
  3. Make any other necessary configuration changes to your AMI.
  4. Save your AMI.
  5. When you deploy the AMI, as part of your node provisioning or as part of the Amazon User Data script run the following commands:
    sudo cloudsight setup --deploy-key=<your deploy key> --ruleset="Base Rule Set" --agent_type=i --hostname=`hostname`
    sudo update-rc.d cloudsight enable
    sudo service cloudsight start

    Replace <your deploy key> with your Distributed Cloud AIP Linux 1.x series Agent deploy key. The Agent now registers and starts when your client boots.

Deploy the Distributed Cloud Containerized Agent 1.x Series

For more information, including an overview of the containerized Agent, see Deploy the Distributed Cloud AIP Containerized Agent.

In the Agent 1.x series, the containerized Agent runs as a privileged container. By operating as a privileged container, the containerized Agent can talk to the kernel and other containers to ensure host security is adequately monitored.

The containerized Agent 1.x series has feature parity with the host-based Agent, with one exception: the customer must create a configuration file for the containerized Agent, as the deploy script does not automatically create one.

Since customers deploy the containerized Agent in containerized environments, Distributed Cloud AIP does not support direct commands to the Agent. Do not use the following commands for the Agent 1.x series:

    • sudo cloudsight start
    • sudo clousight stop
    • sudo cloudsight restart
    • sudo cloudsight config

To deploy the containerized Agent for 1.x series:

  1. Open the Command Line.
  2. Paste the deploy command copied from threatstack/ts-docker, append the following, and press ENTER:
    :latest

    Example:

    docker pull threatstack/ts-docker:latest
  3. Create a configuration file. Distributed Cloud AIP suggests creating it in /etc/ts-agent and name it "ts-config". Ensure the “configuration”: variable is present. Ensure your deploy key is present. Example below. Or Download a sample ts-config.json file.
    {

    "deploy-key": "<your deploy key>",

    "agent_type": "i",

    "ruleset": "Base Rule Set, Docker Rule Set",

    "configuration": {

    "enable_containers":1,

    "log_level": "info"

    }

    }

    Warning

    If you do not provide a configuration file, or if you provide a misconfigured configuration file, then the deployment of the containerized Agent will not work.

  4. To find your image ID, type the following command and press ENTER:
    docker images

    Note

    Running this command displays a list of docker repositories, along with their respective IMAGE ID.

  5. Copy the image ID.
  6. The following command is the deploy command for the container. It should be entered as one block. Type or copy and paste the following command and press ENTER:
    sudo docker run -it -d \
    -e THREATSTACK_CONFIG_PATH="/etc/ts-agent/ts-config" \
    --name=ts-docker \
    --privileged \
    --network=host \
    --pid=host \
    --cap-add=AUDIT_CONTROL \
    --cap-add=AUDIT_READ \
    --cap-add=NET_ADMIN \
    --cap-add=SYS_ADMIN \
    -v /:/threatstackfs/ \
    -v /var/run/docker.sock:/var/run/docker.sock
    <paste IMAGE ID here>

    Replace <paste IMAGE ID here> with the image ID copied in step 5.

    The containerized Agent successfully deploys to the Docker environment.

  7. Confirm the containerized Agent deployed correctly.
    1. Log into Distributed Cloud AIP and view the new server.
    2. Log into the container and run the following command:
      sudo docker exec <container name> cloudsight status
      Replace <container name> with the container name specified in step 6 for --name.
Deploy Containerized Agent 1.x Series Using Kubernetes DaemonSet
For more information, including an overview and list of deployment prerequisites, see Distributed Cloud AIP Kubernetes Deployment.
  1. Create a configuration file for the Distributed Cloud AIP containerized Agent.
    1. In the /etc/ directory, create a ts-agent folder.
    2. Download a sample ts-config.json file.

      Warning

      If you do not provide a configuration file, or if you provide a misconfigured configuration file, then the deployment of the containerized Agent will not work.

    3. Make any changes to the sample file necessary for your environment.
    4. In the /etc/ts-agent folder, save the file as “ts-config.json”.
  2. Create the Kubernetes DaemonSet file.
    1. Download a sample Distributed Cloud AIP Kubernetes DaemonSet .yaml file.
    2. Make any changes to the sample file necessary for your environment.
    3. Include your unique deploy key.
    4. Save the file as “TSKubernetesDaemonSet.yaml”.
  3. Map the configuration file for the Distributed Cloud AIP containerized Agent to the Kubernetes DaemonSet.
    1. Open the Command Line.
    2. Type or copy and paste the following command and press ENTER:
      kubectl create configmap ts-config --from-file=ts-config.json

      This command maps the Distributed Cloud AIP containerized Agent configuration file – ts-config – to the Kubernetes DaemonSet.

  4. Deploy the Distributed Cloud AIP containerized Agent using the Kubernetes DaemonSet.
    1. Go to the Command Line.
    2. Type or copy and paste the following command and press ENTER:
      kubectl create -f TSKubernetesDaemonSet.yaml
  5. Confirm the containerized Agent deployed correctly.
    1. Log into Distributed Cloud AIP.
    2. Ensure events display as expected.
Re-register the Distributed Cloud AIP Linux 1.x Agent

If you log into Distributed Cloud AIP and the Servers page does not display the expected number of servers, you may need to re-register your Agent(s). If you see an "Agent has been revoked. Shutting down" message in /opt/threatstack/cloudsight/logs/cloudsight.log, then you need to re-register your Agent.

Prerequisites

  • Administrator access to your Amazon Web Service (AWS) account
  • Access to the Distributed Cloud AIP console
  • Your deployment key, which can be found in Settings > Keys

Instructions

  1. In the Command Line, type the following command and press ENTER:
    sudo cloudsight stop
  2. Type the following command and press ENTER:
    sudo rm /opt/threatstack/cloudsight/config/.secret
  3. Do one of the following:
    1. To re-register your Agent with the Distributed Cloud AIP Base Rule Set, type the following command and press ENTER:
      sudo cloudsight setup --deploy-key=<your deploy key>

      Replace <your deploy key> with your deployment key.

    2. To re-register your Agent with a different Distributed Cloud AIP ruleset, type the following command and press ENTER:
      sudo cloudsight setup --ruleset=”<ruleset name>
      --deploy-key=<your deploy key>

      Replace <your deploy key> with your deployment key. Replace <ruleset name> with the Distributed Cloud AIP Ruleset name, such as HIPAA.

      Note

      You can specify multiple rulesets for an Agent by including comma separated ruleset names in the ruleset parameter.

  4. Type the following command and press ENTER:
    sudo cloudsight start
Agent 1.x Health Check

The Distributed Cloud AIP Agent uses a two-way connection to communicate with the Distributed Cloud AIP backend. To check the connection health, run the cloudsight status command. The results indicate the health of Distributed Cloud AIP’s processes and connection.

The cloudsight status command returns the following connection information:

Distributed Cloud AIP Cloud Sight RUNNING or STOPPED with Process ID
Distributed Cloud AIP Connection CONNECTED, DISCONNECTED, or WAITING FOR READY*
Distributed Cloud AIP Audit Collection Service RUNNING or STOPPED with Process ID
Distributed Cloud AIP File Integrity Monitoring  RUNNING or STOPPED with Process ID

*If the Agent does not receive a READY response within 30 seconds, then it attempts to reconnect.

Additionally, as a host-based Agent feature, you can run the cloudsight setup -q command to return quiet output results.

Note

You cannot run the cloudsight status -q command in a 1.x container.

The command returns the following information:

Connections 0 (running) or 1 (stopped)
Processes 0 (active) or 1 (disconnected)
Troubleshoot Linux Agent 1.x Performance Issues

Performance Issue(s) Information Gathering

This section is designed to help you gather information if you experience performance issues with the Distributed Cloud AIP Agent. If you gather this information at the time of the incident, then future troubleshooting steps that may put your production environment in a risky state will be reduced.

Gather information to answer the following questions:

  • How many hosts/workloads are affected by the issue?
  • What is the AWS instance type for the affected workload?
  • Is the Distributed Cloud AIP Agent installation on the affected systems/workloads new or existing?
  • Have there been any recent changes to the affected systems/workloads? (Examples: Kernel upgrade; a new version of Java for our workload; etc.)
  • Is your Distributed Cloud AIP Agent deployment scripted? If so, what script tools do you use? Have there been any recent changes to the script?
  • Does the affected environment(s) use the Distributed Cloud AIP Agent File Integrity Monitoring (FIM) or Container Monitoring features?
  • Are there any security tools besides the Distributed Cloud AIP Agent installed on the affected environment(s)?
  • On an unaffected host with a similar workload, what is the typical resource utilization?
  • How can Distributed Cloud AIP best replicate the workload on the affected environment(s) in our environments for further troubleshooting?

Host Reproduction Testing

This section is designed to help you reproduce and troubleshoot the reported performance issue without affecting your production environment. Perform these steps in the listed order in your development environment:

  1. Install the most recent Distributed Cloud AIP Agent version.
  2. Prior to enabling the Distributed Cloud AIP Agent, change Distributed Cloud AIP Agent logging to “debug”:
    sudo cloudsight config log_level=debug

    For more information, please see FAQ: Change level of logging on Agents.

  3. Install the Distributed Cloud AIP support tools, using the following command:
    apt-get install threatstack-agent-support

    For more information, please see Diagnostic Tools and Support Logs.

  4. Start the Distributed Cloud AIP Agent and monitor its usage, using the following command:
    sudo cloudsight setup name=value

    For more information, please see Deploy Distributed Cloud AIP Linux Agent 1.x Series.

  5. If the Distributed Cloud AIP Agent is found in distress, then run the support tools:
    cd /opt/threatstack-agent-support
    sudo ./diagnostics.sh

    For more information, please see Diagnostic Tools and Support Logs.

  6. From the Command Line, gather the following information:
    • auditctl -s
    • auditctl -l
    • htop or top output of a host in trouble.
  7. Forward these captures and the gpg output of the diagnostics to Distributed Cloud AIP Support for review.
  8. Disable FIM tracking and restart the Distributed Cloud AIP Agent (This disables our service which adds inotify and fanotify watches on files which are configured to be monitored. We expect this to reduce resource load, especially during times when files are accessed or changed).
    • sudo cloudsight config disable_fim=1
    • sudo cloudsight restart
Agent 1.x Configuration Arguments

In the Command Line, type the cloudsight help command and press ENTER. All available commands display:

  • cloudsight [command] [options] setup – Set up Distributed Cloud AIP Agent
  • version – Version Information
  • help – Application Usage

Control commands:

  • start – Start Distributed Cloud AIP Agent
  • stop – Stop Distributed Cloud AIP Agent
  • restart – Restart Distributed Cloud AIP Agent
  • status – Show Distributed Cloud AIP Agent status
Handshake Unauthorized Error

Symptoms

Run the cloudsight setup command. Distributed Cloud AIP Agent services do not start and the following error message displays in the Command Line and is captured in the audit logs:

Error received: handshake unauthorized

The error message means the Agent attempted to connect to the Distributed Cloud AIP backend, but was unable to successfully authenticate.

Issue(s)

There are three potential reaons behind a "handshake unauthorized" error message:

  • The .secret file is corrupt or invalid

    Note

    You can find the .secret file at: /opt/threatstack/cloudsight/config/.secret.

  • Someone revoked the Agent or the Agent went offline, and you tried to rerun the setup commands on the same host without deleting the .secret file
  • Your Amazon Machine Image (AMI) creation did not follow the procedure recommended in Steps for Deploying the Distributed Cloud AIP Agent via AMI

Resolution

  1. Confirm that no Distributed Cloud AIP Agent processes are running on the host.
  2. Delete the /opt/threatstack/cloudsight/config/.secret file.
  3. Run the cloudsight setup command again.
Agent 1.x FAQs
Change level of logging on Agents

Distributed Cloud AIP provides the ability to change the level of logging on your Agents with a simple configuration value.

To change the Agent's logging level:

Open the command line and type the following command:

  • sudo cloudsight config log_level=[value]

where value is replaced with one of the following options:

  • fatal- The Agent logs a message when it can no longer proceed without a restart due to an error condition.
  • error- The Agent logs a message when an important task cannot be completed.
  • warn- The Agent logs a message when an unexpected event occurs that may impair the agent.
  • info- The Agent logs a message for important configuration settings and the start and stop of major software subsystems. Info is the default logging level.
  • debug- The Agent logs a message for key state information useful to software engineers for debugging, as well as the start and stop of minor software subsystems.
  • trace- The Agent logs a message with detailed state information useful to software engineers for debugging
Workaround for the Known Linux Limitation with auditd
For an overview with more information, see this FAQ.

To enable raw_log for the Distributed Cloud AIP Agent 1.x:

  1. Navigate to /opt/threatstack/etc/.
  2. In your text editor of choice, open the audit.config.json file.
  3. In "raw_log":, at the same depth as filter, nnsleep, and noop, add the following:
    "/opt/threatstack/cloudsight/logs/tsaudit-raw.log"

    For an example, see below.

  4. Restart the Distributed Cloud AIP Agent with the following command:
    sudo cloudsight restart
  5. On the OS, disable auditd.
  6. Point any application that needs access to auditd data to the raw event stream, in this case /opt/threatstack/cloudsight/logs/tsaudit-raw.log. This prevents non-Distributed Cloud AIP applications from subscribing to the audit socket and causing conflicts with the Distributed Cloud AIP Agent.

Example of raw_log enabled for the Distributed Cloud AIP Agent:

{

"threatstack": {

"auditd": {

"cpumon": {

"inc-by": 300,

"dec-by": 50,

"avg-after": 5,

"ival-sec": 1,

"ival-usec": 0

},

"extra-netinfo": true,

"max-eoe-flush": 100,

"noop": 3,

"nnsleep": 1,

"raw_log":"/opt/threatstack/cloudsight/logs/tsaudit-raw.log",

"filter": "/opt/threatstack/etc/tsauditd.lua",

raw_log Name Rotation

The raw_log naming convention rotates between the name provided in audit.config.json (in the example above, tsaudit-raw.log) and the name with a ".1" appended (using the example above, tsaudit-raw.log.1).

Note

Distributed Cloud AIP renames tsaudit-raw.log to tsaudit-raw.log.1 and then creates a new tsaudit-raw.log file with new content.

Does Distributed Cloud AIP have a hostname limitation? (What does error [36] mean?)

Distributed Cloud AIP has a 62 character hostname limitation. If you exceed the character limit, then you see the following symptoms in the Distributed Cloud AIP platform:

  • The tsfim service crashes repeatedly.
  • In the Agent 1.x series, the output of the cloudsight/logs/threatstack-tsfim.log displays error message: “File name too long [36]” or similar.
Wrong ruleset imported during command line Agent registration

If you register Agents through the Command Line and the wrong rulesets import, then follow these troubleshooting steps or contact support:

  1. Confirm the ruleset name(s) in the organization.
    1. Log into Distributed Cloud AIP.
    2. In the left navigation pane, click Rulesets.
    3. Confirm the names and spelling of each ruleset to import during registration.
  2. Verify your Agent registration script uses the exact same ruleset name(s) as the organization.
    1. Names contain the same spaces.
    2. Names contain the same capitalization.
    3. Names are spelled the same.
    4. Names do not the comma special character in ruleset name(s).
    5. If a - d do not match the organization’s ruleset name(s), then correct the names in the Agent registration script.
  3. In the command line, run the following command to reset your rulesets:

    sudo cloudsight config --ruleset

Note

Use a --ruleset <ruleset name> for each ruleset to associate with the Agent, such as --ruleset HIPAA or --ruleset PCI.

How do I secure my AWS ECS workload with Distributed Cloud AIP?

Deploy the Distributed Cloud AIP host-based Agent – not the containerized Agent – to secure an AWS Elastic Container Service (ECS) environment.

The AWS ECS has an orchestration layer that sits between the host and containers. The orchestration layer allows AWS to manage container instances. However, the orchestration layer does not allow access to the host kernel audit information, which prevents the Distributed Cloud AIP containerized Agent from deploying correctly. As a result, the Distributed Cloud AIP host-based Agent must be used to monitor activity in AWS ECS.

To ensure you receive container events, follow these steps:

  • For Agent 1.x series: go to /opt/threatstack/bin/cloudsight config and set enable_containers=1
How do I secure my AWS EKS workload with Distributed Cloud AIP?

Deploying the Distributed Cloud AIP Agent to Amazon Web Services (AWS) Elastic Container Service for Kubernetes (EKS) can be done with either the host-based or the containerized Agent. Distributed Cloud AIP provides a link to a sample Kubernetes Daemonset for container deployments in this article.

To ensure you receive Kubernetes events, go to /opt/threatstack/bin/cloudsight config and set enable_kubes=1.

Was this article helpful?
0 out of 0 found this helpful