My paper of choice this time was "A Comprehensive Approach to Image Spam Detection: From Server to Client Solution" by Yan Gao, Alok Choudhary and Gang Hua. Since our service will be definitely be exposed to spams and unwanted advertisements, I wanted to explore this issue deeper. Since we are dealing with mostly pictures and drawings, traditional text-based spam detection techniques will be just a smaller portion of our system. Image based detection is a must for our solution.
As the researchers explained, image based spams are intended to circumvent text-based filters, while still delivering the spam message, in a form of a picture. Creating and manipulating an image via scripts are the most common methods of creating image spams, while it is difficult to distinguish image spam from not-spam. (Spammers often use CAPTCHA like distortion to avoid detection.) They discussed, how classifying images itself may lead to a better solution than recognizing the text inside the image and run traditional text-based spam filter on the content. The authors also explained the aspects and possibilities of supervised, semisupervised classifier (not clustering ;) ) and active learning algorithm usage.
The essence of their approach can be summarized as:
We propose a nonnegative sparsity induced similarity measure and apply it for the task of cluster analysis of spam images. The basic proposition we make is that an image should be able to be effectively reconstructed by a small number of other images from the same cluster. This is especially true for spam images because many spam images are generated from a limited number of source images.
Yan Gao; Choudhary, A.; Gang Hua; , "A Comprehensive Approach to Image Spam Detection: From Server to Client Solution," Information Forensics and Security, IEEE Transactions on , vol.5, no.4, pp.826-836, Dec. 2010
doi: 10.1109/TIFS.2010.2080267
They proposed a combined approach to incorporate server and client side detection, in order to get a better detection rate. This approach can be particularly incorporated in our project, as user groups might not be able to cross-reference all images, however our server software can run a spam analyzer in a background thread. Their server side recognition (clustering) is a mathematical similarity-based correlation based, based on the assumption, that junk mail will be delivered in bulk, and even with false negatives, smaller number of junk will be caught on the client-side, and it is still less annoying than the current state. Their client side and active learning approach was also described in details, but my main focus here was the server side, as I am not responsible for client side programming in this project.
Their success rate was actually quite remarkable,and it has a definite potential for usage. As the authors noted, this is no silver bullet, yet it can distinguishably expedite the process of establishing a working framework for spam detection. (Less samples in supervised learning with the yield of more accurate recognition)
Thanks for reading
--
Balazs Makany
Thanks for reading
--
Balazs Makany
No comments:
Post a Comment