2016-06-24 17:34:37 +02:00
|
|
|
The 'mod_pottymouth' ejabberd module aims to fill the void left by 'mod_shit'
|
|
|
|
which has disappeared from the net. It allows individual whole words of a
|
|
|
|
message to be filtered against a blacklist. It allows multiple blacklists
|
2016-08-02 17:52:41 +02:00
|
|
|
sharded by language. The internal bloomfilter can support arbitrary blacklist
|
|
|
|
sizes. Using a large list (say, 87M terms) will slow down the initial server
|
|
|
|
boot time (to about 15 minutes respectively), but once loaded lookups are very
|
|
|
|
speedy.
|
2016-06-24 17:34:37 +02:00
|
|
|
|
2017-07-26 18:56:58 +02:00
|
|
|
Prerequisite bitarray lib:
|
|
|
|
|
|
|
|
mod_pottymouth uses a modified version of the 'etbloom' library that uses
|
|
|
|
'bitarray' to replace 'hipe_bifs'. Ejabberd doesn't handle installing
|
|
|
|
dependences of dependecies quite so well (etbloom being a dep of mod_pottymouth
|
|
|
|
and bitarray being a dep of etbloom), so bitarray needs to be installed manually
|
|
|
|
before installation of mod_pottymouth.
|
|
|
|
|
|
|
|
This is how I got it to work... YMMV.
|
|
|
|
Given $EJABBERD_HOME is the base directory of your ejabberd install:
|
|
|
|
|
|
|
|
mkdir -p $EJABBERD_HOME/erlang-lib/bitarray
|
|
|
|
cd $EJABBERD_HOME/erlang-lib/bitarray
|
|
|
|
clone https://github.com/ferd/bitarray git repo
|
|
|
|
run: /usr/lib/erlang/bin/escript rebar get-deps
|
|
|
|
run: /usr/lib/erlang/bin/escript rebar compile
|
|
|
|
run: /usr/bin/install -c -d /usr/local/lib/bitarray-1.0.0/ebin
|
|
|
|
run: /usr/bin/install -c -d /usr/local/lib/bitarray-1.0.0/priv
|
|
|
|
run: /usr/bin/install -c -m 644 ./ebin/bitarray.app /usr/local/lib/bitarray-1.0.0/ebin/bitarray.app
|
|
|
|
run: /usr/bin/install -c -m 644 ./ebin/bitarray.beam /usr/local/lib/bitarray-1.0.0/ebin/bitarray.beam
|
|
|
|
run: /usr/bin/install -c -m 644 ./priv/bitarray.so /usr/local/lib/bitarray-1.0.0/priv/bitarray.so
|
|
|
|
|
|
|
|
To install mod_pottymouth in ejabberd:
|
2016-06-24 17:34:37 +02:00
|
|
|
|
|
|
|
cd ~/.ejabberd-modules/sources
|
2017-07-26 18:56:58 +02:00
|
|
|
clone the ejabberd-contrib git repo
|
2016-06-24 17:34:37 +02:00
|
|
|
cd mod_pottymouth
|
|
|
|
edit: ./conf/mod_pottymouth.yml
|
|
|
|
|
|
|
|
make sure ejabberd is running
|
|
|
|
run: ejabberdctl module_install mod_pottymouth
|
|
|
|
run: ejabberdctl restart
|
|
|
|
module will be installed in: ~/.ejabberd-modules/mod_pottymouth
|
|
|
|
|
|
|
|
Config file format:
|
|
|
|
|
|
|
|
modules:
|
|
|
|
mod_pottymouth:
|
|
|
|
blacklists:
|
|
|
|
default: /home/your_user/blacklist_en.txt
|
|
|
|
en: /home/your_user/blacklist_en.txt
|
|
|
|
cn: /home/your_user/blacklist_cn.txt
|
|
|
|
fr: /home/your_user/blacklist_fr.txt
|
2016-08-02 17:52:41 +02:00
|
|
|
charmaps:
|
|
|
|
default: /etc/ejabberd/modules/mod_pottymouth/charmap_en.txt
|
|
|
|
en: /etc/ejabberd/modules/mod_pottymouth/charmap_en.txt
|
2016-06-24 17:34:37 +02:00
|
|
|
|
|
|
|
For each language (en,cn,fr,...whatever) provide a full path to a backlist file.
|
|
|
|
The blacklist file is a plain text file with blacklisted words listed one per
|
|
|
|
line.
|
|
|
|
|
2016-08-02 17:52:41 +02:00
|
|
|
You can also provide an optional 'charmap' for each language. This allows you
|
|
|
|
to specify simple substitutions that will be made on the fly so you don't need
|
|
|
|
to include those permutations in the blacklist. This keeps the blacklist small
|
|
|
|
and reduces server startup time. For example, if you included the word:
|
|
|
|
'xyza' in the blacklist, adding the following substitutions in the charmap
|
|
|
|
would filter permutations such as 'XYZA', 'xYz4', or 'Xyz@' automatically.
|
|
|
|
|
|
|
|
charmap format:
|
|
|
|
|
|
|
|
[
|
|
|
|
{"X", "x"},
|
|
|
|
{"Y", "y"},
|
|
|
|
{"Z", "z"},
|
|
|
|
{"@", "a"},
|
|
|
|
{"4", "a"}
|
|
|
|
].
|
|
|
|
|
2016-06-24 17:34:37 +02:00
|
|
|
Gotchas:
|
|
|
|
|
|
|
|
The language will be looked up by whatever value is passed in the xml:lang
|
|
|
|
attribute of the xml message. So, any xml:lang value to be supported will need
|
|
|
|
a corresponding entry/blacklist in the config file. If xml:lang is missing,
|
|
|
|
the 'default' entry in config will be used.
|
|
|
|
|
|
|
|
For xml:lang attribute docs, see:
|
|
|
|
http://wiki.xmpp.org/web/Programming_XMPP_Clients#Sending_a_message
|
|
|
|
|
2016-08-02 17:52:41 +02:00
|
|
|
Blacklist helper
|
2016-06-24 17:34:37 +02:00
|
|
|
|
2016-08-02 17:52:41 +02:00
|
|
|
Thinking of a bunch of swear words and all the permutations can be tough. We made
|
|
|
|
a helper script to take a bare wordlist and generate permutations given a
|
|
|
|
dictionary of substitution characters: https://github.com/madglory/permute_wordlist
|
2016-06-24 17:34:37 +02:00
|
|
|
|
|
|
|
Tip of the hat:
|
|
|
|
|
|
|
|
This mod makes use of the excellent 'etbloom' module:
|
|
|
|
https://github.com/erlangtoolbox/etbloom
|