Test my PC Security
Is your computer secure? Is the firewall that you have installed working properly? Those questions are rather hard to answer even for users with lots of PC experience. Many users simply use the Windows Firewall and think their computer is secure because most experts say that you do need a firewall but do not make any recommendations and if they recommend a firewall they are often biased or try to sell you a product that they own or receive commission for.
Test My PC Security is a website that is offering several Firewall and Host Intrusion Prevention System Leak Tests that you can use to test local firewalls. The interesting aspect of this website is however that you can view test results online. And with test results I do not mean your own but those of other users. It does contain a ranking for Windows XP and Windows Vista with the Firewall that passed most (or even all) tests on the top and those that did not do that well at the bottom.
Ratings range from Excellent to Unacceptable with Comodo Firewall receiving the only Excellent rating. Comodo on the other hand seems to be the company that created the Test My PC Security website. I don't think that they are cheating openly about the results but it could be that they designed the tests that way that their firewall would do very well. Now, I'm not an expert on the topic and it just seems a little bit strange, that is all I have to say.
Update: Please note that the site has not been updated in a while. The global test results are for instance only listing Windows Vista and XP, but not newer versions of Windows that Microsoft released in the meantime. While it may still be a great experience to run the tests on your system, the results may not be as easy to check as they were previously.
Advertisement
Thanks just got rid of a pain in the neck virus thanks to your blog
Hello Matias —
I can see why that would be a concern, so let me set the record straight on that score as well.
We don’t determine/control which list of tests are used – that is determined by the (independent) users. Users “nominate†which tests should be included and which ones do not work well and should be deleted. This is an iterative, community process which Comodo has no say over. Nor can we “change” community’s decision as to which tests to post or delete.
Net, net – I want to thank you all for pointing out where the credibility of this site might be suspect. We intend to revise the site to address these concerns.
Watch this space.
Judy Shapiro
@rarst
I think it is important who designed the test. Just to take one example you could have 100 different tests, but only publish 50 of them, the ones which put your system in the best light. I am not in any way saying this was done in this case.
The most glaring example of this would be in the pharma industry, by only funding those scientist who tend to get positive results the published efficacy of a drug can be improved.
Hi Judy,
I guess it’s all in the ranking, assuming tests are driven impartially, and I have no reason to not assume that. But ranking goes further than testing, in this way that it elaborates a conclusion of overall quality, and that point is in my opinion to be the exclusivity of a tester not involved with his own product in the list.
But I understand your point of view, clearly stated as it is.
Hmmm… I am sorry but most of comments are really weird “they must be cheating! I don’t know anything about security but if they own site they must be cheating!”
1. It’s common for security test to incorporate files detected as harmful either by signature or behaviour analysis. That’s the point – to check if your defences up to catching it.
2. It doesn’t mean crap who designed particular test if your defence failed it. What would you people prefer – reliable test from people who are strongly affiliated with security or 3rd party test from people who hardly know what they are dealing with?
3. How are you even imaging cheating in leak tests? Hole either open or closed, there are no holes that are cheated. So comodo passes some test your firewall fails? That doesn’t mean comodo is best of the best, but it does mean your firewall has holes.
Yes, that testing site also has an advertising role. So has this blog – do you all badmouth Martin on how he dares to put ads on page? Does it somehow prevents you from getting useful info from here?
Then wth all the flame to site that does exactly same – gives you something useful with some ads. You don’t like ads – don’t click them. You don’t like test results – don’t look at them. But you still can get all tasty info/tools/whatever.
That’s called internet.
Hi Transcontinental –
If I understand you correctly – you question the model of having a security software company “running†a testing site because of the potential perceived conflict of interest (kinda of like the fox guarding the chicken coup). We agree in principle :).
That’s why we took great pains to totally separate this from our normal operations. It has its own forum and wiki. It has its own processes for testers and posting results. The site has already helped competitive solutions fix bugs detected by the community of testers.
The main reason we decided to “sponsor†this site is because our large base of users (4mm+) expressed frustration at the lack of a comprehensive, independent repository of tests. Currently, the main testing sites are commercial and their agenda is (understandably) in generating revenue.
Comodo is known as being user driven and that’s why we launched this site. The need for an independent, non commercial testing resource was brought up to us a lot. So we decided to listen and do something about it. And listening to customers is always a good thing.
Judy
Hello,
Just downloaded at checked with kapersky, here is what it shows
File contains malware and cannot be Disinfected
Malware:
exploit.Win32.Copycat.b
it also shows:
Trojan.Win32.Agent.pc
not-virus.Hoax.win43.spycar.a
Also for the best check of your firewall visit http://www.grc.com and choose the leaktest program.
I agree with Transcontinental, and even if you are the best than it is a bad thing to show off. It’s like Microsoft making a test where the result is that IE7 is safer than anything else. Or a politician saying that he is the best to vote on. It can be the truth but when people know that the test is made by the # 1 ranker, none of them will trust the results even when its 500% clear that the outcome is an immutable fact.
P.S. When you make such a test you don’t want to let them know that you made it, right? And that’s why Comodo should remove the test.
Judy Shapiro, I’m afraid I cannot agree on the fact a company involved in firewall security leads tests concerning firewall security. This has nothing to do with honesty, but only with procedures which are not acceptable. I’d even say, it is not in my opinion a smart move, considering that many may trust less Comodo after than before, because oil and water just don’t go together!
Now, this apart, I’ve been told Comodo was an excellent product, and I remain convinced that the traditional approaches are the best in order to get an opinion.
I see that Agnitum Outpost is the # 4,maybe for the blocking but the software itself on vista is crap. When I installed (the full version) on my pc, it caused slowdowns of my mouse on such an anoying way that I coudn’t work anymore. So the solution was ” bye bye outpost” since then I haven’t tried any firewall only programm because of this experience.
Now I have Kaspersky Internet Security for a long time now and its really great, and has saved me from trojans and bad scripts on hacked sites. So with such a internet security suite I don’t see a reason to have another firewall besides it.
Hello again —
Certain security software, including antivirus applications like Kaspersky, work on a system of blacklisted signatures in order to identify and then block the malware. If a certain file is on this blacklist, then your AV will flag it as a virus. Put in other words, this is a false positive because the AV vendor has added one (or more) of the tests in the zip file to it’s blacklist. Indeed, our testers encounter this issue when testing software that is both firewall and antivirus. They wish to test the firewall part of it but the AV is (incorrectly) flagging it as a virus. Fortunately, most AV applications allow the tester to create an exception for the test in question to allow it to run.
I hope that helps.
Judy Shapiro
For a thorough, informative review of the leading firewall contenders (esp. Comodo and Online Armor), see this recent article from Scot’s Newsletter Blog:
http://blog.scotsnewsletter.com/2008/03/24/the-best-firewall-software-of-2008-online-armor/
Dan
Judy thanks for clarifying this issue that I had with the website. Can you comment on the detection of viruses and trojans in the test suite ?
Hi — My name is Judy Shapiro from Comodo and I wanted to set the record straight.
You are correct in that Comodo provides the infrastructure for this site. But let me emphasize, we can not and do not affect the results of any of the tests. The results reflect the real world – and Comodo Firewall has been proven to excel.
It is important to note that most of the tests are not Comodo tests – but industry recognized tests that we gather into one place so community members can access them conveniently.
Our pride of place in the results is because our firewall is really that good – not because we created tests that we can pass. Our results are validated by the industry recognition this solution has won since its launch – PC Magazine Editor’s Choice, CNET 5 star rating, Matousec excellent rating…
This site is really meant to be a resource for the community and the industry. Community members get accurate information and the producers of software get great input. In fact, within the first few weeks of launch, one of the testers identified a bug in OnlineArmour’s firewall.
We are proud of our product – but this site has not been created to promote our product. It was solely created to address the need for better, more accurate testing information.
Judy Shapiro
VP, Corp Communications
Jonathan you know that those tests could be considered malware because they try to circumvent the firewall ? Maybe that’s what Kaspersky picked up. Care to share what it actually was that was found ?
The file (completetests.zip) contains malware, this is rather ironic, due to the fact that the software is supposed to detect security issues, maybe malware is there way of testing the system… Oh well, thank goodness for Kapersky!