Today marked the beginning of our deep dive into SIEM (Security Information and Event Management) and EDR (Endpoint Detection and Response) systems. From now on, we’ll be focusing a lot on Splunk and the critical skill of log analysis.
We started with an overview of how a SIEM works. In essence, a SIEM collects and aggregates data from various sources within a network (like firewalls, servers, and applications), then analyzes this data for signs of potential security threats. It helps security teams monitor, detect, and respond to incidents more efficiently by providing real-time insights.
One of the important concepts we explored was Correlation Rules. These are used in SIEM systems to link seemingly unrelated events and detect complex attacks. For example, if an unusual login event is followed by an access request to a sensitive file, a correlation rule could trigger an alert, flagging this as suspicious activity.
After getting familiar with SIEMs, we moved on to Splunk, a widely-used platform for searching, monitoring, and analyzing machine-generated big data via a web-style interface. Splunk dashboards allow users to visualize and interact with their data in real-time, helping them identify patterns and anomalies. It’s an essential tool in cybersecurity for log management and incident response.
We then spent some time learning how to work with logs—system logs, application logs, and security logs. Logs are records of events that happen within a system, and they can vary widely in format. Common types of logs include:
- Apache logs: These contain information about web server traffic, errors, and requests.
- System logs: They capture system-level events like boot sequences, hardware failures, or resource usage.
- Security logs: These track authentication attempts, access to sensitive files, and potential security incidents.
One of the most important commands I learned today was this simple yet powerful line in the terminal:
cut -d ' ' -f 1 apache.logThis command is used to extract specific columns from a log file, in this case, pulling the first field (often the IP address) from an Apache log file. The -d flag sets the delimiter (in this case, a space), and -f 1 tells the command to retrieve the first field. It was an eye-opener for how easy it can be to filter out the data I need when working with large log files. I also learned about the Sort and Uniq commands, which allow me to:
- Sort: Organize log entries in ascending or descending order based on specific fields (like time or IP address).
- Uniq: Remove duplicate entries, which helps clean up logs and makes analysis more efficient.
After that, we got into Regular Expressions (regex) for grep, a powerful search tool. Regular expressions are patterns used to match character combinations in strings, and they are essential for organizing and cleaning up log files. By using regex with grep, I can search for specific patterns, like all failed login attempts or unusual IP addresses, within a log file. This makes analyzing large volumes of logs much more manageable.
It was another proud day in my cybersecurity journey! After class, I plan to finish the OWASP Top 10 vulnerabilities room and continue moving forward on my path!
