A Crawler-based Digital Forensics Method Oriented to Illegal Website

There are a large number of illegal websites on the Internet, such as pornographic websites, gambling websites, online fraud websites, online pyramid selling websites, etc. This paper studies the use of crawler technology for digital forensics on illegal websites. First, a crawler based illegal website forensics program is designed and developed, which can detect the peripheral information of illegal websites, such as domain name, IP address, network topology, and crawl key information such as website text, pictures, and scripts. Then, through comprehensive analysis such as word cloud analysis, word frequency analysis and statistics on the obtained data, it can help judge whether a website is illegal.