How to Spoof JA3 Signature?
I am using python requests library to make HTTP calls. However website bot detection is using JA3 fingerprint verification and blocking me.
Is there any way I can spoof the JA3 signature.
Category Archives: web-crawler
How to develop a vulnerability scanner from the scratch
As a Student, We need to do tools from scratch to understand how the tools work.
For that, I have planned to develop a website scanner that scans all pages of websites to identify the vulnerabilities of the websites.
how can we do this ki… Continue reading How to develop a vulnerability scanner from the scratch
Why is my web site being scanned for license.txt, and should I be worried?
Lately I am seeing multiple daily 404s for variations of “license.txt”, e.g., “wordpress/license.txt”, “blog/license.txt”, “old/license.txt”, “new/license.txt”. Here’s a little snippet of slightly redacted logfile to illustrate:
5.189.164… Continue reading Why is my web site being scanned for license.txt, and should I be worried?
Burpsuite Crawling – Display Function Location on Result
Is there is any way for me to display Burpsuite crawling result as Function Location?
What i mean by function location is like for example:
URL: x.com/functions/showdata
Function Location: User > Profile > Details
So function location i… Continue reading Burpsuite Crawling – Display Function Location on Result
Strange request googlebot
I found one strange request from googlebot on my site.
[REQUEST_URI] => /@,:()?._-s
[REMOTE_ADDR] => 66.249.64.139
And the IP is Location: Mountain View, United States – 66.249.64.139 is a likely static assigned Corporate IP addre… Continue reading Strange request googlebot
Questions about SOCKS5 security
I’m planning to start a distributed crawler in order to avoid common limitations imposed by servers/CDN like rate limit, region filter, and others.
My idea is to have a central server and multiple agents that will run on dif… Continue reading Questions about SOCKS5 security
Why fingerprint a browser if a fingerprint can be replayed?
I’m facing an issue with rampant scraping and abuse on a website that costs me a good chunk of money to maintain. So, I have been looking to implement a few solutions, and apparently these solutions fingerprint the client in … Continue reading Why fingerprint a browser if a fingerprint can be replayed?
How and why is my site being abused?
I own a popular website that allows people to enter a phone number and get information back about that phone number, such as the name of the phone carrier. It’s a free service, but it costs us money for each query so we show … Continue reading How and why is my site being abused?
Does a searchable public database exist of (hostname; ip) mappings?
This question is not about the trivial usage of the forward/reverse DNS.
Getting the IP of a hostname is trivial (DNS), and using reverse DNS, also we can get (typically) a single hostname of an IP.
However, particularly for massive http… Continue reading Does a searchable public database exist of (hostname; ip) mappings?
Spider (web scrawling) vs Force Browsing
I’m scanning websites with OWASP ZAP and noticed that the Spider attack and Force Browse attack return somewhat different results.
What are the differences between Web Spiders (Web Crawlers) and Force Browsing (of directorie… Continue reading Spider (web scrawling) vs Force Browsing