{"id":261,"date":"2024-01-13T08:46:28","date_gmt":"2024-01-13T16:46:28","guid":{"rendered":"https:\/\/francisco.x10.bz\/blog\/?p=261"},"modified":"2025-01-13T09:44:45","modified_gmt":"2025-01-13T17:44:45","slug":"honeypot-installation-revisited-ditching-splunk-for-mysql","status":"publish","type":"post","link":"https:\/\/defrancisco.us\/blog\/index.php\/2024\/01\/13\/honeypot-installation-revisited-ditching-splunk-for-mysql\/","title":{"rendered":"Honeypot Installation Revisited: Ditching Splunk for MySQL"},"content":{"rendered":"

Back on July 22, 2023, I wrote a blog post<\/a> describing how to install the Cowrie honeypot and use Splunk Enterprise as its data repository. Although proprietary, Splunk is free if you keep your data volume under 500 MB per day. That solution worked fine for about six months, but then one day the honeypot experienced a spike in traffic of 2.5 TB, which allowed me to see the ugly side of the freemium business model and Splunk’s heavy-handed approach to user management: "pay up or be blocked for a month."<\/em> As both options were unacceptable to me, I decided to migrate my data to a free and open-source solution that wouldn’t impose data caps. I chose and implemented my current Cowrie analytics environment based on MySQL and the MySQL REST Service. In other words, I ditched Splunk, moved to mySQL, and never looked back. The notes below provide detailed instruction on how to install and configure the new environment:<\/p>\n

\"Simplified<\/a><\/p>\n

Installing and configuring a Cowrie and MySQL-based honeypot environment is pretty straightforward. Below you can find the steps I took, in case you find the information useful:<\/p>\n

[Skip table of contents<\/a>]<\/p>\n

    \n
  1. The Honeypot<\/a>\n
      \n
    1. Choose the honeypot host<\/a><\/li>\n
    2. Install the system dependencies<\/a><\/li>\n
    3. Create a user account<\/a><\/li>\n
    4. Get the Cowrie code<\/a><\/li>\n
    5. Set up a Python virtual environment<\/a><\/li>\n
    6. Configure Cowrie<\/a><\/li>\n
    7. Customize Cowrie<\/a><\/li>\n
    8. Forward listening ports<\/a><\/li>\n
    9. Start Cowrie<\/a><\/li>\n<\/ol>\n<\/li>\n
    10. The Data Repository<\/a>\n
        \n
      1. Wrangle the Data<\/a><\/li>\n
      2. Install MySQL<\/a><\/li>\n
      3. Create a MySQL database for Cowrie<\/a><\/li>\n
      4. Install Apache, PHP and phpMyAdmin<\/a><\/li>\n
      5. Import the Cowrie data into the MySQL database<\/a><\/li>\n
      6. Configure the MySQL REST Service (MRS)<\/a><\/li>\n
      7. Verify the MRS Core API<\/a><\/li>\n
      8. Reference: The MRS JSON filter grammar<\/a><\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

        Part 1: The Honeypot<\/a><\/h2>\n

        I chose Cowrie as the key component of my unwanted traffic detection infrastructure. Cowrie is a superb medium\/high interaction honeypot designed to log brute-force attempts and shell interactions launched by attackers over both SSH and Telnet. Cowrie is very popular among both researchers and enthusiasts due to an optimal combination of rich capabilities and ease of use. It is open-source and is backed by an active community led by Michel Oosterhof<\/a>, the project\u2019s maintainer, creator, and main developer.<\/p>\n

        1. Choose the honeypot host<\/a><\/h3>\n

        You need to start by choosing a Linux system where to install the honeypot. Since Cowrie is very efficient in its resource consumption, I opted for a tiny Raspberry Pi 400<\/a> computer as the Cowrie host.<\/p>\n

        2. Install the system dependencies<\/a><\/h3>\n

        Install the system dependencies on the Cowrie host:<\/p>\n

         $ sudo apt-get install git python3-virtualenv libssl-dev libffi-dev build-essential libpython3-dev python3-minimal authbind virtualenv \n<\/code><\/pre>\n
        3. Create a user account<\/a><\/h3>\n
        Installing a user without a password is not an absolute requirement, but it is recommended by the Cowrie authors:<\/p>\n
        $ sudo adduser --disabled-password cowrie\nAdding user 'cowrie' ...\nAdding new group 'cowrie' (1002) ...\nAdding new user 'cowrie' (1002) with group 'cowrie' ... \nChanging the user information for cowrie\nEnter the new value, or press ENTER for the default\nFull Name []:\nRoom Number []:\nWork Phone []:\nHome Phone []:\nOther []:\nIs the information correct? [Y\/n]\n\n$ sudo su - cowrie\n<\/code><\/pre>\n
        4. Get the Cowrie code<\/a><\/h3>\n
        Clone the cowrie<\/code> project from GitHub:<\/p>\n
        $ git clone http:\/\/github.com\/cowrie\/cowrie\nCloning into 'cowrie'...\nremote: Counting objects: 2965, done.\nremote: Compressing objects: 100% (1025\/1025), done.\nremote: Total 2965 (delta 1908), reused 2962 (delta 1905), pack-reused 0 \nReceiving objects: 100% (2965\/2965), 3.41 MiB | 2.57 MiB\/s, done.\nResolving deltas: 100% (1908\/1908), done.\nChecking connectivity... done.\n\n$ cd cowrie\n<\/code><\/pre>\n
        5. Set up a Python virtual environment<\/a><\/h3>\n
        Technically speaking, this step is not needed, but it is highly recommended to ensure that package updates on the Cowrie host system will not cause incompatibilities with the honeypot operation:<\/p>\n
        $ pwd\n\/home\/cowrie\/cowrie\n\n$ python -m venv cowrie-env\nNew python executable in .\/cowrie\/cowrie-env\/bin\/python \nInstalling setuptools, pip, wheel...done.\n<\/code><\/pre>\n
        After you install the virtual environment, activate it and install required packages:<\/p>\n
        $ source cowrie-env\/bin\/activate\n(cowrie-env) $ python -m pip install --upgrade pip\n(cowrie-env) $ python -m pip install --upgrade -r requirements.txt \n<\/code><\/pre>\n
        6. Configure Cowrie<\/a><\/h3>\n
        The Cowrie configuration is stored in the cowrie\/etc\/cowrie.cfg file<\/code>. To run the honeypot with a standard configuration, there is no need to change anything. By default, Cowrie accepts traffic over SSH. I wanted the honeypot to also accept traffic over Telnet, send the data to Splunk (more on this later), and change the default ports 22 and 23, so I modified the configuration file as follows:<\/p>\n
        [telnet]\nenabled = true\n...\n[output_splunk]\nenabled = true\n...\n[proxy]\nbackend_ssh_port = 2022\nbackend_telnet_port = 2023 \n<\/code><\/pre>\n
        I also wanted to change the default user configurations and the list of credentials accepted to login to the remote shell. These changes are made by modifing the cowrie\/etc\/userdb.txt file. Each line in the file consists of three fields separated by the : character, where:<\/p>\n