How We Cut Spam by 75% with Abstract API, Fail2Ban & ModSecurity
Tired of spam cluttering your inbox and wasting your resources? So were we. From dodgy sign-ups to fake comments, spam was a constant nuisance. One of the biggest challenges businesses face online is dealing with spamwhether it's in contact forms, comment sections, or sign-up pages. Spam not only clutters your inbox but also exposes your site to potential security risks. But weve tackled the problem head-on, and heres how we did it by integrating AbstractAPI's Email Validation Service.
Stage 1: Cutting Spam with Real-Time Email Validation
Spam often originates from fake or disposable email addresses. To stop these in their tracks, we integrated AbstractAPIs Email Validation Service.
Why Email Validation Works
Spam often originates from fake or disposable email addresses. By validating emails in real-time, we can ensure only legitimate users with real email addresses are allowed through our forms. AbstractAPI checks if the email is syntactically correct, verifies the domain, and even checks if an email is valid, active, and not disposable.
Our Approach: Real-Time Filtering
- Initial Email Validation: Every email submitted through our forms is sent to AbstractAPI. The service quickly validates if the email is real, active, and not disposable.
- Blocking Fake Emails: If an email fails validation, it's immediately blocked, preventing spam submissions.
- Monitoring API Usage: We monitored our API usage closely to ensure efficiency and cost-effectiveness.
Development Team's Research & Suggestions
Through our research and discussions with the development team, we identified additional layers of security to further block spam effectively:
- Combining IP and Email Validation: The team suggested leveraging CleanTalks anti-spam block list alongside AbstractAPI. By first checking the spam score of an IP address and then validating the email, we create a two-step verification process.
- Threshold-Based Filtering: To reduce unnecessary API calls, we decided to apply a threshold-based approach:
- If an IP has a spam rate below 0.3, allow the submission without further checks.
- If an IP has a spam rate between 0.3 and 0.5, validate the email using AbstractAPI.
- If an IP has a spam rate above 0.5, block the submission immediately.
- Rate Limiting & Caching: The team recommended implementing rate limiting to prevent excessive API requests and caching validation results to reduce repeated checks for the same user.
- User Feedback & Monitoring: We are actively monitoring user reports and logs to fine-tune the thresholds, ensuring that legitimate users are not blocked while effectively filtering out spam.
Results: A Clear Decline in Spam
Before using AbstractAPI, our site faced nearly 500 spam submissions per day. After integrating AbstractAPI and implementing additional security layers, spam was reduced to around 100 submissions per day. This drastic drop demonstrates the power of real-time email validation in cutting down spam.
Why We Chose AbstractAPI
One of the key reasons we chose AbstractAPI was its affordable pricing structure and free trial, which allowed us to test the API before committing to a larger plan. The Email Validation API and Phone Validation API were the primary tools we evaluated for our business needs. AbstractAPI offers different pricing plans based on API call limits, ensuring flexibility and scalability as our business grows.
- Starter Plan: 10,000 API calls/month for $19/month (or $12/month if billed annually).
- Basic Plan: 25,000 API calls/month for $39/month (or $36/month if billed annually).
- Standard Plan: 50,000 API calls/month for $69/month (or $63/month if billed annually).
- Business Plan: 100,000 API calls/month for $99/month (or $91/month if billed annually).
Stage 2: Enhancing Security with Fail2Ban and ModSecurity
While real-time email validation significantly reduced spam, we took our security measures even further by fine-tuning Fail2Ban and ModSecurity. These two powerful security tools allowed us to block malicious traffic, prevent brute-force attacks, and filter out unwanted spam requests at the server level.
Why We Chose Fail2Ban & ModSecurity for Spam Prevention
- Blocking Spam Bots & Malicious IPs with Fail2Ban: Fail2Ban is an intrusion prevention system that automatically bans IP addresses exhibiting suspicious activitylike excessive failed login attempts or spam form submissions. Since many spam attacks originate from the same bad IP ranges, Fail2Ban helped us block these threats dynamically without manual intervention.
- Stopping Malicious Requests with ModSecurity: ModSecurity is a Web Application Firewall (WAF) that monitors and filters HTTP traffic. We configured it to detect common spam patterns, such as bots trying to post junk content, probing for vulnerabilities, or accessing non-existent files (404 attacks).
By combining Fail2Bans dynamic IP blocking with ModSecuritys request filtering, we established a robust, multi-layered defense against spam.
Fine-Tuning Fail2Ban for Spam Filtering
To maximize Fail2Bans efficiency, we implemented custom jails and rules to specifically target spam bots.
Configuration (/etc/fail2ban/jail.local)
[apache-spam]enabled = truefilter = apache-spamlogpath = /var/log/apache2/access.logmaxretry = 3bantime = 86400findtime = 3600
- maxretry = 3 If an IP submits 3 spammy form requests, it gets banned.
- bantime = 86400 The IP is banned for 24 hours.
- findtime = 3600 The 3 failed attempts must occur within 1 hour to trigger a ban.
Fine-Tuning ModSecurity to Block Spam Requests
ModSecurity works as a server-side firewall that monitors HTTP requests and blocks spam traffic before it even reaches our application.
Step 1: Enabling ModSecurity on Apache/Nginx
sudo apachectl -M | grep security # For Apachesudo nginx -V 2>&1 | grep modsecurity # For Nginx
If ModSecurity was not enabled, we installed it:
sudo apt install libapache2-mod-security2 -y # Apachesudo apt install libnginx-mod-http-modsecurity -y # Nginx
Adjusting ModSecurity Rules to Reduce Spam
SecRule REQUEST_METHOD "POST" "id:10001,phase:1,deny,status:403,msg:'Spam detected'"
Blocking Known Spam Bots (/usr/share/modsecurity-crs/rules/custom-spam-rules.conf)
SecRule REQUEST_HEADERS:User-Agent "@pmFromFile /etc/modsecurity/spam-bots.txt" \ "id:10002,phase:1,deny,status:403,msg:'Known spam bot blocked'"
Final Results: A 75% Drop in Spam
Before: ~500 spam-related API calls per day.
After Fail2Ban: ~277 calls per day.
After ModSecurity: ~100 calls per day.
By combining real-time email validation, Fail2Ban, and ModSecurity, weve created a multi-layered spam defence system thats both effective and efficient.
Whats Next? AI and Honeypots
While weve seen incredible results, were not stopping here. Our next steps include:
- AI-Based Detection: Using machine learning to spot emerging spam trends.
- Honeypots: Tricking bots into revealing themselves.
The Takeaway
