Site Categorization in Dell SonicWALL Content Filtering Service

Direct2Dell

The Official Dell Corporate Blog

Site Categorization in Dell SonicWALL Content Filtering Service

Dell, through the SonicWALL acquisition, has been in the internet content filtering business for over a decade.  During that time the world has gone from fewer than a 100,000 registered URLs to more than a billion.  Our job is to categorize those sites and give IT administrators control over their network. In those 10 years, we’ve had very few complaints.  Yes, we sometimes rate a site in one way and either that site’s emphasis changes over time, or, we may make a categorization mistake.  It does happen.  In other cases a website generates little traffic and we don’t come across it so our system responds as “Not Rated".  On the whole though, we’ve been very good and we’ve received a lot of positive credit for it.  

Over the last few days, we’ve had a different experience.

A school had a policy to block a category of sites rated as Politics/Advocacy Groups at their site using our content filtering product. It’s important to note that our product does not come with that category turned on. The school actively turned it on. The result was a student was prevented from doing some key research. Further, the policy at this school allowed “Not Rated” sites to be accessed. Most school IT administrators block the Not Rated category since millions of new *** and malware sites come online each month and it is very important to block them until they can be rated. The combination of blocking Politics/Advocacy Groups had a bad outcome. The student found he was blocked from many conservative political sites but could get to many sites he described as liberal that had the category of “Not Rated.” The student couldn't do appropriate research and the combination of the block/allow policy lead to a perception that we had a political bias. Not only that, someone jumped to the conclusion that we somehow rated conservative sites and somehow gave liberal sites a “Not Rated” category. 

Amusingly, logic should have dictated a different conclusion. “If” we had somehow deliberately rated liberal sites “Not Rated” through some conspiracy, then it would mean more liberal sites would generally be blocked than conservative sites. Why? Because most IT Administrators naturally block the Not Rated category due to the dangers of exposing networks to malware and other problems. Kind of funny. If this conspiracy were true, it would actually be more anti-liberal in most networks. But guess what? This has become a run-away story.  

I was asked how we can filter conservative, liberal or other political categories. Here is what gets rated and how:

  • The ratings categories in the Dell SonicWALL Content Filtering Service do not have the granularity to block access based on conservative, liberal or any other political bias.  We provide a higher level category that blocks all “Political and Advocacy Groups” that does not take into account any political affiliation.  This category is not turned on by default.  A user must actively choose to turn it on.  The category is not able to single out any party bias.
  • An automated numerical frequency algorithm is used to determine the order of the queue for previously unrated sites.  Sites that receive the highest traffic are placed at the top of the queue, and conversely sites that have low traffic volume are lower in the queue.
  • We also provide users with a tool to assess how any particular URL is rated.  If a user finds a site that is not rated or is improperly rated, he or she can enter that URL into the tool and it goes for immediate review and rises to the top of the queue regardless of frequency.   There is no “selection” criteria used to determine what gets rated other than numerical frequency, or end user requests/submissions. 
  • The Dell SonicWALL Content Filtering Service allows administrators to block any site that has not been rated by providing a category called “Not Rated”.  It is important to note that Not Rated means that we have not yet had the opportunity to rate the URL.  It does not mean that we have reviewed the site and decided to rate it “Not Rated”.   Many organizations block the “Not Rated” category since millions of new URLs are introduced monthly and will not make it into the rating queue until there is sufficient traffic or an end user submits the site for rating. Not Rated should be blocked since millions of new malware sites appear monthly.

This is a tricky business and it is not a perfect science. But we do our best to help IT administrators by providing the tools so that they can use their networks the way they want to.  We don’t make any judgments, we simply try to give tools.  And if we make mistakes, we correct them. But, unfortunately, the subject area is not so exciting that it can support a full-blown conspiracy.

Illustration of how a content filtering client works

To post a comment login or create an account

Comment Reminder

Unrelated comments or requests for service will be unpublished. Please post your technical questions in the Support Forums or for direct assistance contact Dell Customer Service or Dell Technical Support.. All comments must adhere to the Dell Community Terms of Use.

  • Great article.

  • I am surprised at Dell not correcting the site categories instead of a how it works article.  If a republican, pro-life, pro-traditional family site is categorized as "political/advocacy group" then the democrat, lgbt, pro-abortion groups should be the same as the default instead of "Not Rated".   Whether intentional or not, it gives the perception of showing bias towards certain groups or beliefs as common sense shows they are all political/advocacy groups.  I would feel the same if it was reversed.  Outside of that your diagram does a good job showing how the content filtering works.

  • awesome!

  • I'm a SonicWall user and I was very disappointed in reading this. I am surprised at Dell not correcting the site categories instead of a how it works article.   Whether intentional or not, it gives the perception of showing bias towards certain groups or beliefs as common sense shows they are all political/advocacy groups.

  • We provide multiple tools so anyone can request a rating or re-rating and it is done in 24 hours. In this particular case, we deliberately held on re-rating the sites the student identified so we could researched based on the original data (I know I'll get a lot of cynicism on that one, but true.) And contrary to what is said below, yes, both pro life and pro choice are BOTH in the Advocacy category. There is run-away misinformation at this point so you'll need to spend a few minutes looking at real data and drawing your own conclusions. Later today, I'll post data regarding the URLs in question that I think you will find interesting.  

  • [I am surprised at Dell not correcting the site categories instead of a how it works article.]

    Perhaps Dell had someone who is good at writing articles writing this, while the folks good at rating sites were busy rating. It's called "multitasking."

    And the "Not Rated" sites can't be "corrected." "Not Rated" doesn't mean someone gave the site that rating, it means that no one has rated them yet. And they haven't been rated because very few people have visited them. How else, besides site popularity, would you set the rating priority?

  • The student, Andrew Lampart, said his initial discovery was that the National Rifle Association’s website was blocked, but not anti-gun websites.  But the more websites Lampart tried to reach, the stranger it got.

    Seems to me that all the anti-gun websites would fall into the "Not Rated" category.

    The he said “I immediately found out that the State Democrat web site was unblocked but the State GOP web site was blocked,” Lampart told FoxCT.

    Same problem - clearly both are political and both with high visibility.

    Lampart checked out websites related to abortion.  He discovered that the National Right to Life website was blocked, but not Planned Parenthood or NARAL Pro-Choice America’s websites.

    This is particularly disconcerting.  Both are clearly advocacy groups - sadly, the latter, is supported by public schools while the former is shunned.  Why would one get through the filter and not the other, especially when it's PP - they have been around on the net advocating for abortion and contraceptive services since the start of the Internet as a common tool.  This does not match up with Mr. Sweeney's explanation.

    Lampart had similar results when checking out religious websites. Christianity.com and the Vatican’s web site were both blocked, but Islam-guide.com was not.

    Just as unsettling and with the same comment as the PP statement.  I don't believe it.  The Vatican has had a website for a very long time and is certainly not political or an advocacy group.  There's clearly more work that needs to be done by this filtering system.

    According to The Daily Caller, each of the blocked sites was struck down by the schools SonicWall filter which labeled them as impermissible “Political/Advocacy Groups” sites.

  • None of this explains why the Connecticut GOP site was banned, but not the Connecticut Democrat site.

  • My belief is that from what I have just read on your website, the school actually is not at fault on this one, and  the blame does go to Dell. Granted the Politics/Advocacy Groups was turned off and the school turned it on. However, the school did not determine what websites went into that group. Why would it expect that site to be so biased? Also, your explanation that some of the liberal sites were not-rated because they are new or receive virtually no visitors does not hold water as it implies that you did not rate Planned Parenthood and Connecticut Democrat sites - all well known and certainly visited sites. However, you did block NRA and Connecticut Republican sites.

  • Patrick, did you ever leave the URL information that you said you would be posting on 6/20?  I'd like to give Dell the benefit of the doubt, but student access to both sides of any story is too important to leave it to an artful flowchart.

  • Visit cfssupport.sonicwall.com to view ratings. Visit sites before you make a judgement and also consider the difficulty in categorization. And yep, we make some mistakes from time to time so submit the mistakes. But when looking for a BIAS, don’t take shortcuts. Fox, Examiner, and Republican American all did follow up stories and dug deeper and found “much a do about nothing”.  Compare like for like sites.  And correlate visitation (a key in this) via tools like www.alexa.com so if you see it being Not Rated, see if it gets traffic.  And be sure to spell the URLs exactly correctly - 4 of the URLs we received from the school were misspelled which means they are “Not Rated” since we don’t have data on them.  That is not a bias, that is bad spelling.  And use logic. The fundamental irony is being ignored.  Not Rated SHOULD be blocked since that is how malware entities get malware in – through the release of millions of new URLs. Plus, factor in that fundamental mistakes in the “research” in the story. Example, Blocking of http://www.teaparty.org/ was sited as evidence of a Liberal bias. But guess what?  That rating was mistakingly NOT Politics/Advocacy so not blocked by Policy/Advocacy.  Summary:  No, not bias. What there was?  Multiple low trafficked sites that had not been rated (which I’ll put into the queue now that the debate is winding down) + the inverse policy on Not Rated (which would have inverted the conclusions) + some bad research used as evidence + several mis-ratings (out of millions).