F u t u r a G T S   I n t e r n e t   S e c u r i t y
D a n s G u a r d i a n    P h r a s e l i s t s

DansGuardian Phraselist Updates

Here you can find the latest development phraselists. Beware that while our testing shows that these existing lists generate fewer false positives and higher catch rates, the new lists may generate more false positives. Please send feedback if you find that phrases are causing false positives for you.

October 23rd 2007 Update

You can download here in two formats: Zip File or TarBall

.
Key Changes:
* Added new weighted phraselists categories: translation, music
* Small updates to various languages in the pornography phraselists
* Several small improvements to other lists
* Added and reorganized additional bannedregexpurllist entries

Note that some of the changes/lists are ALPHA quality.  Some lists are empty, some incomplete - especially the new categories. I'd like feedback and additions if possible especially for the following:
* peer2peer, news, sports, games and webmail lists
* foreign language lists - do they overblock other languages - ie portuguese blocking spanish or english pages?
Please direct any comments, criticism and suggestions to phrasemaster@dansguardian.org.



October 8th 2007 Update

You can download the new lists from:
http://contentfilter.futuragts.com/phraselists/phraselistsoct8.zip
http://contentfilter.futuragts.com/phraselists/phraselistsoct8.tar.gz


Key Changes:
* Added new weighted phraselists categories: conspiracy, domainsforsale, idtheft, secretsocieties, selflabelling, travel, upstreamfilter
* Lists now include #listcategory: "listname"
* Small updates to various languages in the pornography phraselists
* Updates and changes to the proxy list
* Many updates to the sports lists
* Many, many small improvements to other lists
* Added additional bannedregexpurllist entries

Note that some of the changes/lists are ALPHA quality.  Some lists are empty, some incomplete - especially the new categories. I'd like feedback and additions if possible especially for the following:
* peer2peer, news, sports, games and webmail lists
* foreign language lists - do they overblock other languages - ie portuguese blocking spanish or english pages?
* SpanishPornography list

Please direct any comments, criticism and suggestions to phrasemaster@dansguardian.org.



May 31st 2006 Update

You can download the new lists from:
http://contentfilter.futuragts.com/phraselists/phraselistsmay31.zip
http://contentfilter.futuragts.com/phraselists/phraselistsmay31.tar.gz


Key Changes:
***Almost too many to List***
* Added new weighted phraselists categories: forums
* Added new languages for pornography: Malay, Japanese, Chinese, Norwegian, Russian
* Added content to the previously empty SpanishPornography list
* Added phrases to the goodphrases list
* Many changes to the proxy and malware lists
* Many, many improvements to other lists - removed overblocking phrases
* Added additional bannedregexpurllist entries

Note that some of the changes/lists are ALPHA quality.  Some lists are empty, some incomplete - especially the new categories. I'd like feedback and additions if possible especially for the following:
* peer2peer, news, sports, games and webmail lists
* foreign language lists - do they overblock other languages - ie portuguese blocking spanish or english pages?
* SpanishPornography list

Please direct any comments, criticism and suggestions to phrasemaster@dansguardian.org.



August 10th 2005 Update

You can download the new lists from:
http://contentfilter.futuragts.com/phraselists/phraselistsaug10.zip
http://contentfilter.futuragts.com/phraselists/phraselistsaug10.tar.gz


Key Changes:
* Added Peer2Peer weighted phrases
* Added phrases to the goodphrases list
* Many, many small improvements to lists - removed overblocking phrases


Note that some of the changes/lists are ALPHA quality.  Some lists are empty, some incomplete - especially the new categories. I'd like feedback and additions if possible especially for the following:
* peer2peer, news, sports, games and webmail lists
* foreign language lists - do they overblock other languages - ie portuguese blocking spanish or english pages?
* Spanish Pornography list

Please direct any comments, criticism and suggestions to phrasemaster@dansguardian.org.



May 31st 2005 Update

You can download the new lists from:
http://contentfilter.futuragts.com/phraselists/phraselistsmay31.zip
http://contentfilter.futuragts.com/phraselists/phraselistsmay31.tar.gz


Key Changes:
* Added goodphrases_danish file - thanks to Frederik Dannemare
* Added porn_danish file - thanks to Frederik Dannemare
* Fixed more overblocking problems especially in the French porn list


Note that some of the changes/lists are ALPHA quality.  Some lists are empty, some incomplete - especially the new categories. I'd like feedback and additions if possible especially for the following:
* news, sports, games and webmaillists
* foreign language lists - do they overblock other languages - ie portuguese blocking spanish or english pages?
* SpanishPornography list

Please direct any comments, criticism and suggestions to phrasemaster@dansguardian.org.



May 12th 2005 Update

You can download the new lists from:
http://contentfilter.futuragts.com/phraselists/phraselistsmay12.zip
http://contentfilter.futuragts.com/phraselists/phraselistsmay12.tar.gz


Key Changes:
* Removed several duplicate phrases
* Fixed several phrases with no score
* Added goodphrases_portuguese file - thanks to Allan Gomes
* Added badwords_portuguese file - thanks to Allan Gomes
* Moved some phrases from /porn/weighted_portuguese to badwords_portuguese - thanks to Allan Gomes
* Fixed accents on several portuguese files - thanks to Allan Gomes
* Fixed more overblocking problems


Note that some of the changes/lists are ALPHA quality.  Some lists are empty, some incomplete - especially the new categories. I'd like feedback and additions if possible especially for the following:
* news, sports, games and webmaillists
* foreign language lists - do they overblock other languages - ie portuguese blocking spanish or english pages?
* SpanishPornography list

Please direct any comments, criticism and suggestions to phrasemaster@dansguardian.org.



May 2nd 2005 Update

Here is an update on the status of the weighted phrase lists.  Substantial changes have been made and they are ready for some testing.  Thank you to those who contributed lists and suggestions.

You can download the new lists from:
http://contentfilter.futuragts.com/phraselists/phraselistsmay2.zip
http://contentfilter.futuragts.com/phraselists/phraselistsmay2.tar.gz

Key Changes:
* Added MANY new lists - see below
* Changed the list file naming structure to weighted_[language] and banned_[language]
* Changed the organization of the weightedphraselist file
* Changed the organization of the bannedphraselist file
* Split content in the exceptionphrase and goodphrases lists to prevent conflict with sports, webmail and news lists.
* Added terms to the bannedregexpurllist file for proxies and nudism
* Added notes to the tops of many files for easier reference/tracking when editing

Note that some of the changes/lists are ALPHA quality.  Some lists are empty, some incomplete - especially the new categories. I'd like feedback and additions if possible especially for the following:
* news, sports, games and webmail lists
* foreign language lists - do they overblock other languages - ie portuguese blocking spanish or english pages?
* Spanish Pornography list

Please direct any comments, criticism and suggestions to phrasemaster@dansguardian.org.

New Lists:

badwords_dutch
badwords_german
badwords_spanish
chat_italian
gambling_portuguese
games
gore_portuguese
illegaldrugs_portuguese
intolerance_portuguese
malware
news
nudism
personals_portuguese
porn_dutch
porn_spanish
porn_portuguese
porn_italian
proxies
sport
violence_portuguese
webmail
weapons_portuguese
** plus several portuguese banned lists

Examples of words removed from existing lists:

< foto ><foto> in several lists generates too many false positives
< adult> in italian generates too many false positives.
<sperm> in italian generates false positives - should be < sperm> < sperm >?
<chat><10> in italian generates too many false positives.
< video ><10> in italian generates too many false positives.



------------------------------------ New May 2nd Weighted Phrase List File ------------------------------------

# NOTE: New lists are commented out as ALPHA or BETA depending on how much the
# lists have been tested.
# ALPHA - Brand new - no testing has been done
# BETA - Relatively new - tested in at least one location
#

#Good Phrases (to allow medical, education, news and other good sites)
.Include</etc/dansguardian/phraselists/goodphrases/weighted_general>
.Include</etc/dansguardian/phraselists/goodphrases/weighted_news>

#Pornography
.Include</etc/dansguardian/phraselists/pornography/weighted>
.Include</etc/dansguardian/phraselists/pornography/weighted_dutch> #ALPHA#
.Include</etc/dansguardian/phraselists/pornography/weighted_french>
.Include</etc/dansguardian/phraselists/pornography/weighted_german>
.Include</etc/dansguardian/phraselists/pornography/weighted_italian>
.Include</etc/dansguardian/phraselists/pornography/weighted_portuguese>
.Include</etc/dansguardian/phraselists/pornography/weighted_spanish> #ALPHA#
.Include</etc/dansguardian/phraselists/nudism/weighted>

#Bad Words - swearing
.Include</etc/dansguardian/phraselists/badwords/weighted_dutch> #BETA#
.Include</etc/dansguardian/phraselists/badwords/weighted_french>
.Include</etc/dansguardian/phraselists/badwords/weighted_german> #ALPHA#
.Include</etc/dansguardian/phraselists/badwords/weighted_spanish> #ALPHA#

#Drugs
#.Include</etc/dansguardian/phraselists/drugadvocacy/weighted>
#.Include</etc/dansguardian/phraselists/illegaldrugs/weighted>
#.Include</etc/dansguardian/phraselists/illegaldrugs/weighted_portuguese>
#.Include</etc/dansguardian/phraselists/legaldrugs/weighted>

#Violence and intolerance
#.Include</etc/dansguardian/phraselists/intolerance/weighted>
#.Include</etc/dansguardian/phraselists/intolerance/weighted_portuguese>
#.Include</etc/dansguardian/phraselists/gore/weighted>
#.Include</etc/dansguardian/phraselists/gore/weighted_portuguese>
#.Include</etc/dansguardian/phraselists/violence/weighted>
#.Include</etc/dansguardian/phraselists/violence/weighted_portuguese>
#.Include</etc/dansguardian/phraselists/weapons/weighted>
#.Include</etc/dansguardian/phraselists/weapons/weighted_portuguese>

#Chat
#.Include</etc/dansguardian/phraselists/chat/weighted>
#.Include</etc/dansguardian/phraselists/chat/weighted_italian>

#Webmail
#.Include</etc/dansguardian/phraselists/webmail/weighted> #ALPHA#
#Note that if you enable the webmail weighted list you should also disable
#the "exception_email" list in the exceptionphraselist file.

#Productivity
#.Include</etc/dansguardian/phraselists/gambling/weighted>
#.Include</etc/dansguardian/phraselists/gambling/weighted_portuguese>
.Include</etc/dansguardian/phraselists/warezhacking/weighted>
#.Include</etc/dansguardian/phraselists/personals/weighted>
#.Include</etc/dansguardian/phraselists/personals/weighted_portuguese>
#.Include</etc/dansguardian/phraselists/games/weighted> #ALPHA#
#.Include</etc/dansguardian/phraselists/sport/weighted> #ALPHA#
#.Include</etc/dansguardian/phraselists/news/weighted> #ALPHA#

#System Management and Security
#.Include</etc/dansguardian/phraselists/malware/weighted> #ALPHA#
#.Include</etc/dansguardian/phraselists/proxies/weighted> #BETA#

Web Content Filtering by DansGuardian
virgin rain forest ringtone dreamcast dreamcast dreamcast fresh investigation fresh investigation fresh investigation fresh investigation guitar guitar guitar guitar guitar guitar guitar guitar guitar guitar hormone replacement hormone replacement hormone replacement hormone replacement hormone replacement hormone replacement investigative documentary investigative documentary investigative documentary investigative documentary lyrics by artist