New securityonion-setup package resolves several issues

I've updated the securityonion-setup package to resolve several issues.  The updated package version is securityonion-setup - 20120912-0ubuntu0securityonion99 and it has been tested by the following (thanks!):
Matt Gregory
David Zawdie
JP Bourget

Issue 463: sosetup: prompt for ELSA log_size_limit
https://code.google.com/p/security-onion/issues/detail?id=463

Issue 470: sosetup: Add verbiage to ELSA screen about running on sensors
https://code.google.com/p/security-onion/issues/detail?id=470

Issue 474: sosetup: increase default query_timeout in /etc/elsa_web.conf
https://code.google.com/p/security-onion/issues/detail?id=474

Issue 388: sosetup: configure MySQL to create an innodb file per table to prevent ibdata1 growing indefinitely
https://code.google.com/p/security-onion/issues/detail?id=388

Issue 416: sosetup: increase default MySQL open-files-limit
https://code.google.com/p/security-onion/issues/detail?id=416

Screenshots

Setup now prompts for ELSA log_size_limit

Setup sets ELSA log_size_limit as requested by user

Setup now sets ELSA query_timeout to 10000

Setup now configures MySQL with better defaults

MySQL now creates an innodb file per table

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

Feedback
If you have any questions or problems, please use our security-onion mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list:
http://groups.google.com/group/security-onion

We also need help testing new packages:
http://groups.google.com/group/security-onion-testing

Thanks!

New NSM and Setup packages available

I've updated our NSM and Setup packages to resolve a few issues:

Issue 429: nsm_server_clear needs latest Squert database updates
https://code.google.com/p/security-onion/issues/detail?id=429

Issue 451: nsm_sensor_clean should purge old files in /nsm/bro/extracted
https://code.google.com/p/security-onion/issues/detail?id=451

Issue 454: Disabling PADS agent blocks PRADS and results in no SANCP
records flowing
https://code.google.com/p/security-onion/issues/detail?id=454
(thanks to Kevin Branch for the patch)

Issue 435: Setup should allow you to set PF_RING min_num_slots
https://code.google.com/p/security-onion/issues/detail?id=435

Issue 446: Setup should delete /var/lib/sphinxsearch/data/binlog*
https://code.google.com/p/security-onion/issues/detail?id=446

Issue 452: Setup phase 2 should populate sniffing interfaces from
/etc/network/interfaces
https://code.google.com/p/security-onion/issues/detail?id=452

Issue 439: /etc/cron.d/sensor-newday updates
https://code.google.com/p/security-onion/issues/detail?id=439

Issue 440: BPF JIT addition to /etc/sysctl.d/10-securityonion.conf
https://code.google.com/p/security-onion/issues/detail?id=440

The new packages are as follows:
securityonion-nsmnow-admin-scripts - 20120724-0ubuntu0securityonion66
securityonion-setup - 20120912-0ubuntu0securityonion92

They have been tested by the following (thanks!):
David Zawdie

Setup now selects interfaces based on /etc/network/interfaces

If you allow Setup to configure /etc/network/interfaces, then it will use that information later to automatically select the proper interface(s) for monitoring:

PF_RING min_num_slots
Setup now creates /etc/modprobe.d/pf_ring.conf to set parameters for the PF_RING kernel module.  If you run Quick Setup, it will just use the default value of 4096 for min_num_slots.  However, if you choose Advanced Setup, you will have the opportunity to change that default value.

If you've already run Setup and want to modify min_num_slots, you can manually create /etc/modprobe.d/pf_ring.conf.  For example, to increase min_num_slots to 65534, do the following:

echo "options pf_ring transparent_mode=0 min_num_slots=65534" | sudo tee /etc/modprobe.d/pf_ring.conf

After creating /etc/modprobe.d/pf_ring.conf, you'll need to reload the PF_RING module as follows (or just reboot):

sudo nsm_sensor_ps-stop
sudo rmmod pf_ring
sudo nsm_sensor_ps-start

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

Feedback
If you have any questions or problems, please use our mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list and IRC channel.  Thanks!

New securityonion-setup package available

I've packaged a new version of Setup that resolves a couple of issues:

Issue 436: sosetup-network: replace ifconfig with iproute2's ip tool
https://code.google.com/p/security-onion/issues/detail?id=436
This patch from Jon Schipp updates sosetup-network so that it uses "ip" instead of "ifconfig".

Issue 441: sosetup-network shouldn't stop network-manager
https://code.google.com/p/security-onion/issues/detail?id=441
The last release of sosetup added a new question to sosetup-network to allow you to modify /etc/network/interfaces and then have the choice to not immediately reboot. If you did this over "ssh -X" it would stop Network Manager which would drop your ssh connection before it could ask if you want to reboot. It should no longer try to stop Network Manager so you should see the final reboot question when running over ssh.

The version number of the new package is securityonion-setup - 20120912-0ubuntu0securityonion90 and it has been tested by the following (thanks!):
Scott Runnels
Matt Gregory
JP Bourget

Updating
The new package is now available in our stable repo.  Please see the following page for full update instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

Feedback
If you have any questions or problems, please use our mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list and IRC channel.  Thanks!

New NSM and Setup packages allow you to enable/disable sensor services

New versions of our securityonion-nsmnow-admin-scripts and securityonion-setup packages are now available that allow you to enable/disable sensor services.  When you run Setup, choosing "Quick Setup" will still default to running all sensor services, but if you choose "Advanced Setup", you'll be prompted to select which sensor processes to enable/disable:

IDS Engine

Bro

http_agent

Argus 

Prads

Full Packet Capture

Your choices are then written into configuration files as follows:

/etc/nsm/securityonion.conf

BRO_ENABLED=yes

/etc/nsm/HOSTNAME-INTERFACE/sensor.conf

PCAP_ENABLED="yes"
PCAP_AGENT_ENABLED="yes"
SNORT_AGENT_ENABLED="yes"
IDS_ENGINE_ENABLED="yes"
BARNYARD2_ENABLED="yes"
PRADS_ENABLED="yes"
SANCP_AGENT_ENABLED="yes"
PADS_AGENT_ENABLED="yes"
ARGUS_ENABLED="yes"
HTTP_AGENT_ENABLED="yes"

Disabling Services after Setup

If you've already run Setup and want to disable a certain sensor service, you can simply stop the running service and then change the corresponding config value from "yes" to "no" to prevent it from restarting the next time the NSM scripts are run.

For example, suppose you access Bro's HTTP logs via ELSA, so you want to disable http_agent to prevent those HTTP logs from being duplicated into the Sguil database.  You would first stop the running http_agent service:

sudo nsm_sensor_ps-stop --only-http-agent

You would then edit /etc/nsm/HOSTNAME-INTERFACE/sensor.conf and change:

HTTP_AGENT_ENABLED="yes"

to:

HTTP_AGENT_ENABLED="no"

to prevent http_agent from restarting the next time the NSM scripts are run.  A quick way to do this for all /etc/nsm/*/sensor.conf files on one box is to use the sed command as follows:

sudo sed -i 's|HTTP_AGENT_ENABLED="yes"|HTTP_AGENT_ENABLED="no"|g' /etc/nsm/*/sensor.conf

Example Screenshots

Stopping the running service

Disabling the service

Service now disabled

Issues Resolved
These updates resolve the following issues:
Issue 312: Update NSM scripts to allow $SERVICE=yes/no in securityonion.conf and/or sensor.conf
Issue 313: Update Setup so that Advanced Setup asks about enabling/disabling individual services
Issue 268: Update NSM scripts so that OSSEC and Bro sections respect --sensor-name option
Issue 351: Update /etc/init/securityonion.conf to start Xplico (controlled by user in /etc/nsm/securityonion.conf)

Thanks
Thanks to Karolis Cepulis for submitting a patch for Issue 268!
Thanks to the following for testing the new packages!
Matt Gregory
Michal Purzynski

Upgrading
The new packages are now available in our stable repo.  Please see our Upgrade page for full upgrade instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

Feedback
If you have any questions or problems, please use our mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list and IRC channel.  Thanks!

Training
Want to learn more about Security Onion?  Sign up for the upcoming 8-hour class in Augusta GA!
http://securityonion.eventbrite.com/

New NSM and Setup packages allow for changing the default 90% disk usage threshold

New versions of our securityonion-nsmnow-admin-scripts and securityonion-setup packages are now available that allow you to change our default 90% disk usage threshold.  When you run Setup, it will still default to purging old logs when disk usage hits 90%.  Choosing "Advanced Setup" will prompt you to specify your own disk usage threshold:

Advanced Setup prompts for disk usage threshold

This setting is then written into /etc/nsm/securityonion.conf as CRIT_DISK_USAGE, where it is sourced by /usr/sbin/nsm_sensor_clean when the hourly cronjob runs.

If you've already run Setup and want to change the default 90% disk usage threshold, you can simply set the value in /etc/nsm/securityonion.conf.

These updates resolve the following issues:
Issue 315: Update NSM scripts so that WARN_DISK_USAGE and CRIT_DISK_USAGE are configurable by user
Issue 358: Update Setup so that Advanced Setup asks about CRIT_DISK_USAGE

Thanks
Thanks to Karolis Cepulis for the nsm_sensor_clean patch!
Thanks to David Zawdie for testing the new package!

Upgrading
The new package is now available in our stable repo.  Please see our Upgrade page for full upgrade instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

Feedback
If you have any questions or problems, please use our mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list and IRC channel.  Thanks!

Training

Want to learn more about Security Onion?  Sign up for the upcoming 8-hour class in Augusta GA!

http://securityonion.eventbrite.com/

New NSM and Setup packages allow for changing the default netsniff-ng PCAP size

New versions of our securityonion-nsmnow-admin-scripts and securityonion-setup packages are now available that allow you to change our default 150MB netsniff-ng PCAP size.  When you run Setup, it will still default to 150MB.  Choosing "Advanced Setup" will prompt you to specify your own PCAP_SIZE:

Advanced Setup prompts for PCAP size

This PCAP_SIZE option is then placed into /etc/nsm/HOSTNAME-INTERFACE/sensor.conf where it is sourced by the NSM scripts when they start netsniff-ng.

If you've already run Setup and want to change the default 150MB PCAP size, you can add the PCAP_SIZE option to /etc/nsm/HOSTNAME-INTERFACE/sensor.conf.  Please note that netsniff-ng accepts the following units for PCAP_SIZE:
KiB
MiB
GiB

So if you want to increase your PCAPs to 500MB, you would add the following option to /etc/nsm/HOSTNAME-INTERFACE/sensor.conf:
PCAP_SIZE=500MiB

Then restart netsniff-ng as follows:

sudo nsm_sensor_ps-restart --only-pcap

Also, I've seen some intermittent cases where pcap_agent fails to start right after running Setup, so I've added a 5-second delay between starting netsniff-ng and starting pcap_agent to help ensure that netsniff-ng is fully initialized.

These updates resolve the following issues:
Issue 341: nsm_sensor_ps-start needs "sleep 5s" between netsniff-ng and pcap_agent
Issue 314: Update NSM scripts so that netsniff-ng pcap size is configurable by user

Thanks
Thanks to JP Bourget for the NSM/Setup patches for setting the PCAP size!
Thanks to the following for testing the new package:
Matt Gregory
Liam Randall
David Zawdie

Upgrading
The new package is now available in our stable repo.  Please see our Upgrade page for full upgrade instructions:
https://code.google.com/p/security-onion/wiki/Upgrade

Feedback
If you have any questions or problems, please use our mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list and IRC channel.  Thanks!

New Setup package configures OSSEC to send alerts to ELSA

Previously, when a user ran Setup and enabled ELSA, they would be able to log into ELSA and view OSSEC *archive* logs (the raw logs received by OSSEC) but they wouldn't be able to view OSSEC *alerts* (created based on OSSEC's analysis of the incoming logs as configured by the OSSEC ruleset).  I've pushed a new Setup package that will configure OSSEC to send alerts to local syslog if the user enables ELSA.  The new package has been tested by Matt Gregory.  Thanks, Matt!

If you've already run Setup and would like to configure OSSEC to send alerts to ELSA, please see:
https://code.google.com/p/security-onion/wiki/OSSECalertsToELSA

Updating
If you're performing a new installation, it's important to update your packages right after you've completed the Ubuntu installer and BEFORE running Setup.  You can initiate the upgrade process using the graphical Update Manager or using the following one-liner:

sudo apt-get update && sudo apt-get dist-upgrade

Upgrade Process

Feedback
If you have any questions or problems, please use our mailing list:
https://code.google.com/p/security-onion/wiki/MailingLists

Help Wanted
If you and/or your organization have found value in Security Onion, please consider giving back to the community by joining one of our teams:
https://code.google.com/p/security-onion/wiki/TeamMembers

We especially need help in answering support questions on the mailing list and IRC channel.  Thanks!