pax_global_header00006660000000000000000000000064135770106440014521gustar00rootroot0000000000000052 comment=5f4ba25406547596fd39406673774d93b06a8e07 dbab-1.3.3/000077500000000000000000000000001357701064400124155ustar00rootroot00000000000000dbab-1.3.3/.gitignore000066400000000000000000000002271357701064400144060ustar00rootroot00000000000000*~ blib/ .build/ _build/ cover_db/ inc/ Build !Build/ Build.bat .last_cover_stats Makefile.old MANIFEST.bak META.yml MYMETA.yml nytprof.out pm_to_blib dbab-1.3.3/LICENSE000066400000000000000000000027531357701064400134310ustar00rootroot00000000000000BSD 3-Clause License Copyright (c) 2013, suntong All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. dbab-1.3.3/Makefile000066400000000000000000000032421357701064400140560ustar00rootroot00000000000000INSTALL = /usr/bin/install -c # Installation directories prefix = ${DESTDIR}/usr exec_prefix = ${prefix} mandir = ${prefix}/share/man docdir = ${prefix}/share/doc/dbab bindir = ${exec_prefix}/sbin libdir = ${DESTDIR}/lib etcdir = ${DESTDIR}/etc astdir = ${etcdir}/dbab ssddir = ${DESTDIR}/lib/systemd/system # Install the systemd unit file if systemd support was detected. ifdef HAVE_SYSTEMD systemdsystemunit_DATA = systemd/dbab.service endif man: cat README.md | ronn > assets/dbab-svr.8 clean: rm -f assets/dbab-svr.8 install: $(INSTALL) -m 755 -d $(bindir) $(INSTALL) -m 755 -d $(mandir)/man8 $(INSTALL) -m 755 -d $(docdir) $(INSTALL) -m 755 -d $(libdir)/init $(INSTALL) -m 755 -d $(etcdir) $(INSTALL) -m 755 -d $(etcdir)/init.d $(INSTALL) -m 755 -d $(astdir) $(INSTALL) -m 755 -d $(ssddir) $(INSTALL) -m 755 bin/dbab-get-list $(bindir) $(INSTALL) -m 755 bin/dbab-add-list $(bindir) $(INSTALL) -m 755 bin/dbab-chk-list $(bindir) $(INSTALL) -m 755 bin/dhcp-add-wpad $(bindir) $(INSTALL) -m 755 bin/dbab-svr $(bindir) $(INSTALL) -m 755 bin/dbab-init-d-script $(libdir)/init $(INSTALL) -m 755 bin/dbab $(etcdir)/init.d $(INSTALL) -m 644 assets/dbab.service $(ssddir) $(INSTALL) -m 644 assets/dbab-svr.8 $(mandir)/man8 $(INSTALL) -m 644 assets/dbab-add-list.8 $(mandir)/man8 $(INSTALL) -m 644 assets/dbab-chk-list.8 $(mandir)/man8 $(INSTALL) -m 644 assets/dbab-get-list.8 $(mandir)/man8 $(INSTALL) -m 644 assets/dhcp-add-wpad.8 $(mandir)/man8 $(INSTALL) -m 644 dbab.md $(docdir) $(INSTALL) -m 644 assets/dbab.addr $(astdir) $(INSTALL) -m 644 assets/dbab.list+ $(astdir) $(INSTALL) -m 644 assets/dbab.list- $(astdir) $(INSTALL) -m 644 assets/dbab.proxy $(astdir) dbab-1.3.3/README.md000066400000000000000000000112361357701064400136770ustar00rootroot00000000000000# dbab(8) -- dnsmasq based ad blocking ## SYNOPSIS # start dbab-svr server /etc/init.d/dbab start # stop dbab-svr server /etc/init.d/dbab stop # get/update ad blocking list /usr/sbin/dbab-get-list # add your own to the ad blocking list /usr/sbin/dbab-add-list ## DESCRIPTION dbab provides a total solution for SOHO service environment, smoothly integrates DHCP, DNS, local caching and Ad blocking into harmony. Ad blocking is done by DNSmasq + Pixelserv, i.e., done at the DNS level -- all requests to ad-sites are blocked right there at DNS. No more user space extensive pattern matching necessary at all. Work for your mobile devices as well. You don't need to install anything to your mobile devices to enjoy the ad-free and speed-up browsing. ## ALTERNATIVES People may also use browsers' adblock-plus extension to block ads, but fewer think over how it works internally. Here is an overview of Adblock Plus from a thousand mile high [1] -- whenever the browser needs to load something, the extension kicks in and do a thorough pattern matching of all known ad urls using regular expressions, then hectically replace all found ad urls with something else. This is done on every page, every load, and every component of the web page, using JavaScript. Thus it is by nature slow and CPU intensive, at least inefficient. There are other alternatives to this, e.g., privoxy, but the concepts are the same. [1] http://adblockplus.org/en/faq_internal ## ADVANTAGES Comparing to other ad-blocking efforts, `dbab` will be super light. Only a few operations are enough to determine and stop the ads. No heavy-lifting (using CPU intensive URL pattern matching) necessary. Thus it will be lighting fast as well. The advantages of using `dbab` are: - **Work at the DNS level**. Leave the web pages intact, without any pattern matching, string substitution, and/or html elements replacing. - **Work for your mobile devices as well**. Were you previously in the dilemma of choosing ads free or slow response for your mobile devices (iphone, ipad, etc)? Now you don't. You don't need to install any thing to your mobile devices for them to enjoy the ad-free browsing experience. Moreover, their browsing speed will increase dramatically on revisited pages/images. - **Serve instantly**. All ads will be replaced by a 1x1 pixel gif image served locally by the `dbab-svr` server. - **Maintenance free**. You don't need to maintain the list of ad sites yourself. The block list can be downloaded from pgl.yoyo.org periodically. If you don't like some of the entries there, you can add-to or remove-from that list easily. ## DBAB-SVR The `dbab-svr` is a super minimal web server / pixelserv, it has one purpose of serving a 1x1 pixel transparent gif file. It can optionally provide the automatic WPAD service as well if so configured. By default it listens on localhost. ## DBAB-GET-LIST The `dbab-get-list` is used to get dnsmasq blocking list from pgl.yoyo.org to be used by DNSmasq. The result is stored as `/etc/dnsmasq.d/dbab.adblock.conf`. You can run it once, or put it in a cron job so as to update the block list periodically. E.g., to update on a weekly basis: ln -s /usr/sbin/dbab-get-list /etc/cron.weekly/ ## DBAB-ADD-LIST You can use `dbab-add-list` to add your own entries to `dnsmasq` blocking list, if the list from pgl.yoyo.org is not sufficient for you. The result is stored as `/etc/dnsmasq.d/dbab.trashsites.conf`. ## DBAB-CHK-LIST The `dbab-chk-list` can help you to check if your own list is already covered by pgl.yoyo.org. ## DHCP-ADD-WPAD he `dhcp-add-wpad` will take the content in `/etc/dbab/dbab.proxy` as the host name of the squid caching server, as well as taking the content in `/etc/dbab/dbab.addr` as the IP address of `dhcp` server, then enable the automatic WPAD service within the system, with the help of the DNS and DHCP server. ## FILES * /etc/dbab/dbab.addr: The IP address that `dbab-svr` listens on. Defaults to localhost. * /etc/dbab/dbab.list-: The entries you want to filter out from the pgl.yoyo.org lists. List sites you still wish to visit there. * /etc/dbab/dbab.list+: The entries you want to add to blocking list on top of the pgl.yoyo.org list, used by `dbab-add-list`. * /etc/dbab/dbab.proxy: The name or IP address of your squid caching server. Defaults to localhost. * /usr/share/doc/dbab/dbab.md: The more detailed introduction and installation guild. ## AUTHOR(S) Copyright: 2013~2015 Tong SUN, suntong001 from users.sourceforge.net License: BSD-3-Clause The pixelserv was originally downloaded from http://proxytunnel.sourceforge.net/files/pixelserv.pl.txt Wrote by Piet Wintjens, with BSD (no advertising clause) license. dbab-1.3.3/assets/000077500000000000000000000000001357701064400137175ustar00rootroot00000000000000dbab-1.3.3/assets/dbab-add-list.8000066400000000000000000000000241357701064400163730ustar00rootroot00000000000000.so man8/dbab-svr.8 dbab-1.3.3/assets/dbab-chk-list.8000077700000000000000000000000001357701064400210672dbab-add-list.8ustar00rootroot00000000000000dbab-1.3.3/assets/dbab-dnsmasq.intranet.conf000066400000000000000000000017771357701064400207610ustar00rootroot00000000000000# == DNS from ISP server=192.168.2.1 # == Dhcp lease (start,end,leasetime) # supply the range of addresses available for lease and optionally # a lease time. If you have more than one network, you will need to # repeat this for each network on which you want to supply DHCP # service. dhcp-range=192.168.2.1,192.168.2.80,48h # == Domain for dnsmasq. this is optional, but if it is set, it # does the following things. # 1) Allows DHCP hosts to have fully qualified domain names, as long # as the domain part matches this setting. # 2) Sets the "domain" DHCP option thereby potentially setting the # domain of all systems configured by DHCP # 3) Provides the domain part for "expand-hosts" domain=EXAMPLE.ORG # == Dhcp hosts. # dhcp-host=00:28:58:3A:EB:A1,192.168.2.20,computer2,infinite # ^ ^ ^ ^ # MAC IP Address hostname lease time # E.g., #dhcp-host=00:16:3e:00:00:01,192.168.0.81,kvm1,8h #dhcp-host=00:16:3e:00:00:02,192.168.0.82,kvm2,8h dbab-1.3.3/assets/dbab-dnsmasq.service.conf000066400000000000000000000047701357701064400205710ustar00rootroot00000000000000# The following options make you a better netizen, since they # tell dnsmasq to filter out queries which the public DNS cannot # answer, and which load the servers (especially the root servers) # uneccessarily. If you have a dial-on-demand link they also stop # these requests from bringing up the link uneccessarily. # Never forward plain names (without a dot or domain part) domain-needed # Never forward addresses in the non-routed address spaces. bogus-priv # By default, dnsmasq will send queries to any of the upstream # servers it knows about and tries to favour servers to are known # to be up. Uncommenting this forces dnsmasq to try each query # with each server strictly in the order they appear in # /etc/resolv.conf #strict-order # By default, when dnsmasq has more than one upstream server # available, it will send queries to just one server. Setting this # flag forces dnsmasq to send all queries to all available servers. # The reply from the server which answers first will be returned to # the original requestor. #all-servers # blocks probe-machines attack stop-dns-rebind rebind-localhost-ok # If you want dnsmasq to detect attempts by Verisign to send queries # to unregistered .com and .net hosts to its sitefinder service and # have dnsmasq instead return the correct NXDOMAIN response, uncomment # this line. You can add similar lines to do the same for other # registries which have implemented wildcard A records. # http://www.thekelleys.org.uk/dnsmasq/docs/setup.html bogus-nxdomain=64.94.110.11 # == Dealing with dnsmasq's own hostname # don't alow dnsmasq to read /etc/hosts, in which dnsmasq's hostname # will most probably listed as 127.0.0.1 no-hosts # instead, configure dnsmasq to use another host-file addn-hosts=/etc/dnsmasq.hosts # == Provide DNS server addresses from this file instead # Google Public DNS (respond time: ~36 msec constantly) server=8.8.8.8 #server=8.8.4.4 #OpenDNS Servers (respond time: ~160 msec initially, ~28 msec afterwards) #server=208.67.222.222 #server=208.67.220.220 # Disabled by default becase OpenDNS will redirect all Google queries to their own search server. Ref: # https://wiki.archlinux.org/index.php/dnsmasq#Prevent_OpenDNS_Redirecting_Google_Queries # Set this (and domain: see below) if you want to have a domain # automatically added to simple names in a hosts-file. expand-hosts # enable dhcp dhcp-authoritative dhcp-option=6,0.0.0.0 # use /etc/ethers for static hosts; same format as --dhcp-host # [] #read-ethers dbab-1.3.3/assets/dbab-get-list.8000077700000000000000000000000001357701064400211012dbab-add-list.8ustar00rootroot00000000000000dbab-1.3.3/assets/dbab-squid.localnet.conf000066400000000000000000000000371357701064400204010ustar00rootroot00000000000000acl localnet src 192.168.0.0/16dbab-1.3.3/assets/dbab-squid.service.conf000066400000000000000000000011621357701064400202400ustar00rootroot00000000000000# Allow access from local networks #include /etc/squid3/dbab-squid.localnet.conf acl localnet src 192.168.0.0/16 # RFC1918 possible internal network http_access allow localnet # refresh pattern for debs and tgzs, 90 days # usage: refresh_pattern [-i] regex min percent max [options] # # By default, regular expressions are CASE-SENSITIVE. To make # them case-insensitive, use the -i option. refresh_pattern -i deb$ 129600 100% 129600 refresh_pattern -i tar.gz$ 129600 100% 129600 refresh_pattern -i tar.xz$ 129600 100% 129600 # cache objects up to 80 MB (virtualbox is ~66M) maximum_object_size 80 MB dbab-1.3.3/assets/dbab.addr000066400000000000000000000000121357701064400154340ustar00rootroot00000000000000127.0.0.1 dbab-1.3.3/assets/dbab.list+000066400000000000000000000000451357701064400155560ustar00rootroot00000000000000dl4all.com filestube.com terapdf.com dbab-1.3.3/assets/dbab.list-000066400000000000000000000000251357701064400155560ustar00rootroot00000000000000googleadservices.com dbab-1.3.3/assets/dbab.proxy000066400000000000000000000000121357701064400157030ustar00rootroot00000000000000localhost dbab-1.3.3/assets/dbab.service000066400000000000000000000004601357701064400161710ustar00rootroot00000000000000# systemd configuration for dbab. -*- conf -*- [Unit] Description=Dnsmasq-Based Ad-Blocker After=network.target Documentation=man:dbab-svr(8) Documentation=https://github.com/suntong/dbab/blob/master/src/dbab.md [Service] ExecStart=/usr/sbin/dbab-svr Type=simple [Install] WantedBy=multi-user.target dbab-1.3.3/assets/dhcp-add-wpad.8000077700000000000000000000000001357701064400210602dbab-add-list.8ustar00rootroot00000000000000dbab-1.3.3/bin/000077500000000000000000000000001357701064400131655ustar00rootroot00000000000000dbab-1.3.3/bin/dbab000077500000000000000000000015271357701064400140100ustar00rootroot00000000000000#! /bin/sh # kFreeBSD do not accept scripts as interpreters, using #!/bin/sh and sourcing. if [ true != "$INIT_D_SCRIPT_SOURCED" ] ; then set "$0" "$@"; # if there is a /lib/init/init-d-script, then source it [ -s /lib/init/init-d-script ] && INIT_D_SCRIPT_SOURCED=true . /lib/init/init-d-script # else, source my private copy to ease backporting [ -e /lib/init/init-d-script ] || INIT_D_SCRIPT_SOURCED=true . /lib/init/dbab-init-d-script fi ### BEGIN INIT INFO # Provides: dbab # Required-Start: $remote_fs $syslog # Required-Stop: $remote_fs $syslog # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: dbab # Description: dnsmasq-based ad-blocking using pixelserv. ### END INIT INFO DESC="dnsmasq-based ad-blocker" DAEMON=/usr/sbin/dbab-svr START_ARGS="--background --make-pidfile" dbab-1.3.3/bin/dbab-add-list000077500000000000000000000010611357701064400155000ustar00rootroot00000000000000#!/bin/sh ##----------------------------------------------------------------------- ## Porgram: dbab-add-list ## Purpose: Add your own entries to dnsmasq blocking list ## Authors: Tong Sun (c) 2013-2019 ## License: BSD license ##----------------------------------------------------------------------- dbab_list=/etc/dbab/dbab.list+ dbabsvr_addr=`cat /etc/dbab/dbab.addr` dbab_conf=/etc/dnsmasq.d/dbab-map.trashsites.conf # Add the trash-sites to DNSmasq blocking sed 's#^.*$#'"address=/&/$dbabsvr_addr#" $dbab_list > $dbab_conf echo File $dbab_conf updated. dbab-1.3.3/bin/dbab-chk-list000077500000000000000000000007361357701064400155250ustar00rootroot00000000000000#!/bin/sh ##----------------------------------------------------------------------- ## Porgram: dbab-get-list ## Purpose: Check if our own list is already covered by pgl.yoyo.org ## Authors: Tong Sun (c) 2013-2019 ## License: BSD license ##----------------------------------------------------------------------- dbab_list=/etc/dbab/dbab.list+ block_list=/etc/dnsmasq.d/dbab-map.adblock.conf # Check if our trash-sites list is already covered grep -Ff $dbab_list $block_list dbab-1.3.3/bin/dbab-get-list000077500000000000000000000014621357701064400155340ustar00rootroot00000000000000#!/bin/sh ##----------------------------------------------------------------------- ## Porgram: dbab-get-list ## Purpose: Get dnsmasq blocking list from pgl.yoyo.org ## Authors: Tong Sun (c) 2013-2019 ## License: BSD license ##----------------------------------------------------------------------- grep -n '^[[:punct:]]*$' /etc/dbab/dbab.list- && { echo "ERROR: Blank line(s) found in /etc/dbab/dbab.list-" printf "\tRemove them before proceeding.\n" exit 1 } dbabsvr_addr=`cat /etc/dbab/dbab.addr` dbab_conf=/etc/dnsmasq.d/dbab-map.adblock.conf # Down the DNSmasq formatted ad block list curl "https://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext" | grep -vFf /etc/dbab/dbab.list- | sed "s/127\.0\.0\.1/$dbabsvr_addr/" > $dbab_conf echo File $dbab_conf updated. dbab-1.3.3/bin/dbab-init-d-script000077500000000000000000000130131357701064400164650ustar00rootroot00000000000000#!/bin/sh # See init-d-script(5) for instructions on how to use this library. #============================================================================= # Define LSB log_* functions. # Depend on lsb-base (>= 3.2-14) to ensure that this file is present # and status_of_proc is working. . /lib/lsb/init-functions # PATH should only include /usr/* if it runs after the mountnfs.sh # script. Scripts running before mountnfs.sh should remove the /usr/* # entries. PATH=/sbin:/usr/sbin:/bin:/usr/bin export PATH is_call_implemented() { command -V $1 > /dev/null 2>&1 } do_usage() { if is_call_implemented do_reload ; then echo "Usage: $SCRIPTNAME {start|stop|status|reload|restart|try-restart|force-reload}" >&2 else echo "Usage: $SCRIPTNAME {start|stop|status|restart|try-restart|force-reload}" >&2 fi } call() { cmd="$1" shift if is_call_implemented ${cmd}_override ; then ${cmd}_override "$@" else ${cmd} "$@" fi } # # Function that starts the daemon/service # # Return # 0 if daemon has been started # 1 if daemon was already running # 2 if daemon could not be started do_start_cmd() { start-stop-daemon --start --quiet ${PIDFILE:+--pidfile ${PIDFILE}} \ $START_ARGS \ --startas $DAEMON --name $NAME --exec $DAEMON --test > /dev/null \ || return 1 start-stop-daemon --start --quiet ${PIDFILE:+--pidfile ${PIDFILE}} \ $START_ARGS \ --startas $DAEMON --name $NAME --exec $DAEMON -- $DAEMON_ARGS \ || return 2 # Add code here, if necessary, that waits for the process to be ready # to handle requests from services started subsequently which depend # on this one. As a last resort, sleep for some time. } do_start() { if is_call_implemented do_start_prepare ; then call do_start_prepare fi [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME" call do_start_cmd case "$?" in 0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;; 2) [ "$VERBOSE" != no ] && log_end_msg 1 ;; esac if is_call_implemented do_start_cleanup ; then call do_start_cleanup fi } # # Function that stops the daemon/service # # Return # 0 if daemon has been stopped # 1 if daemon was already stopped # 2 if daemon could not be stopped # other if a failure occurred do_stop_cmd() { start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 \ $STOP_ARGS \ ${PIDFILE:+--pidfile ${PIDFILE}} --name $NAME --exec $DAEMON RETVAL="$?" [ "$RETVAL" = 2 ] && return 2 # Wait for children to finish too if this is a daemon that forks # and if the daemon is only ever run from this initscript. # If the above conditions are not satisfied then add some other code # that waits for the process to drop all resources that could be # needed by services started subsequently. A last resort is to # sleep for some time. start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 \ $STOP_ARGS \ --exec $DAEMON [ "$?" = 2 ] && return 2 # Many daemons don't delete their pidfiles when they exit. rm -f $PIDFILE return $RETVAL } do_stop() { if is_call_implemented do_stop_prepare ; then call do_stop_prepare fi [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME" call do_stop_cmd case "$?" in 0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;; 2) [ "$VERBOSE" != no ] && log_end_msg 1 ;; esac if is_call_implemented do_stop_cleanup ; then call do_stop_cleanup fi } do_restart() { [ "$VERBOSE" != no ] && log_daemon_msg "Restarting $DESC" "$NAME" call do_stop_cmd call do_start_cmd case "$?" in 0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;; 2) [ "$VERBOSE" != no ] && log_end_msg 1 ;; esac } do_force_reload() { if is_call_implemented do_reload ; then call do_reload else call do_restart fi } # Enable this using # alias do_reload=do_reload_sigusr1 do_reload_sigusr1() { log_daemon_msg "Reloading $DESC configuration files" "$NAME" start-stop-daemon --oknodo --stop --signal 1 --quiet \ --pidfile "$PIDFILE" --exec "$DAEMON" log_end_msg $? } do_status() { status_of_proc "$DAEMON" "$NAME" && return 0 || return $? } if [ "$DEBUG" = "true" ] ; then set -x fi SCRIPTNAME=$1 scriptbasename="$(basename $1)" if [ "$scriptbasename" != "init-d-script" ] ; then script="$1" shift . $script else exit 0 fi NAME=${NAME:=$(basename $DAEMON)} DESC=${DESC:=$NAME} # Do not use pid file if $PIDFILE is 'none'. Otherwise, generate from # $NAME or use the value provided by the init.d script. if [ none = "$PIDFILE" ] ; then PIDFILE= elif [ -z "$PIDFILE" ] ; then PIDFILE=/var/run/$NAME.pid fi # Exit if the package is not installed if [ none != "$DAEMON" ] && [ ! -x "$DAEMON" ] ; then exit 0 fi # Read configuration variable file if it is present [ -r /etc/default/$NAME ] && . /etc/default/$NAME # Load the VERBOSE setting and other rcS variables . /lib/init/vars.sh if [ -t 0 ] ; then # Be verbose when called from a terminal VERBOSE=yes fi case "$1" in start) call do_start ;; stop) call do_stop ;; status) call do_status ;; reload) if is_call_implemented do_reload ; then do_reload else call do_usage exit 3 fi ;; force-reload) call do_force_reload ;; restart) call do_restart ;; try-restart) log_daemon_msg "Trying to restart $DESC" "$NAME" if call do_status > /dev/null 2>&1 ; then call do_restart log_end_msg $? else log_progress_msg "is not running." log_end_msg 1 fi ;; '') call do_usage exit 3 ;; *) if is_call_implemented do_unknown ; then call do_unknown "$1" exit 3 else call do_usage exit 3 fi ;; esac exit 0 dbab-1.3.3/bin/dbab-svr000077500000000000000000000070431357701064400146170ustar00rootroot00000000000000#! /usr/bin/perl -Tw ##----------------------------------------------------------------------- ## Porgram: dbab-svr pixelserv ## Purpose: super minimal webserver serving a 1x1 pixel transparent gif ## Authors: Tong Sun (c) 2013-2019 ## Authors: Originally wrote by Piet Wintjens, date unknown ## License: covered by the new BSD (no advertising clause) license ## Reference: Well House Consultants training course ## http://www.wellho.net/resources/ex.php4?item=p402/miniserver.pl ##----------------------------------------------------------------------- use IO::Socket::INET; my $conffile = "/etc/dbab/dbab.addr"; my $proxyfile = "/etc/dbab/dbab.proxy"; my $crlf = Socket::CRLF; my $pixel = pack( "C*", qw(71 73 70 56 57 97 1 0 1 0 128 0 0 255 255 255 0 0 0 33 249 4 1 0 0 0 0 44 0 0 0 0 1 0 1 0 0 2 2 68 1 0 59) ); my $fh; #-------- conf file --------------- open($fh, "<", $conffile) || die "can't open $conffile: $!"; my $listento = do { local $/; <$fh> }; close($fh) || die "can't close $conffile: $!"; if ( $listento =~ /^([\d.]+)$/ ) { $listento = $1; # $listento now untainted } else { die "Bad listen to address: '$listento'"; } #-------- proxy file --------------- open($fh, "<", $proxyfile) || die "can't open $proxyfile: $!"; my $proxyaddr = do { local $/; <$fh> }; close($fh) || die "can't close $proxyfile: $!"; if ( $proxyaddr =~ /^([\w.]+)$/ ) { $proxyaddr = $1; # $proxyaddr now untainted } else { die "Error opening proxy definition file: '$proxyaddr'"; } my $autoProxy = "function FindProxyForURL(url, host) {". qq| return "PROXY $proxyaddr:3128; DIRECT"; }$crlf|; # Setup and create socket my $sock = new IO::Socket::INET( LocalHost => $listento, LocalPort => '80', Proto => 'tcp', Listen => 30, Reuse => 1 ); if ( !defined($sock) ) { print "error : cannot bind : $! exit\n"; exit(1); } # If the connection is open, but get closed before dbab-svr has # sent the pixel, the SIGPIPE gets send, and dbab-srv will die. # Ignore such signal as we can write to a closed connection anyway. $SIG{'PIPE'} = 'IGNORE'; # Await requests and handle them as they arrive while (1) { my $new_sock = $sock->accept() or next; # set timeout of 2s to avoid blocking the whole program $new_sock->setsockopt(SOL_SOCKET, SO_RCVTIMEO, pack('l!l!', 2, 0)); my %request = (); local $/ = $crlf; #-------- Read Request --------------- while (<$new_sock>) { chomp; # Main http request if (/\s*(\w+)\s*([^\s]+)\s*HTTP\/(\d.\d)/) { $request{METHOD} = uc $1; $request{URL} = $2; $request{HTTP_VERSION} = $3; next; } # print ">$_<\n"; if ( /^$/ ) { last; } } if (defined($request{METHOD}) and $request{METHOD} eq 'GET' and ($request{URL} eq '/proxy.pac' or $request{URL} eq '/wpad.dat') ) { #------- Serve pac/wpad file -------------------- print $new_sock "HTTP/1.0 200 OK$crlf"; print $new_sock "Connection: close$crlf"; print $new_sock "Content-Type: application/octet-stream$crlf$crlf"; print $new_sock "$autoProxy"; } else { #------- Serve pixel file ---------------------- print $new_sock "HTTP/1.0 200 OK$crlf"; print $new_sock "Connection: close$crlf"; print $new_sock "Cache-Control: public, max-age=31536000$crlf"; print $new_sock "Content-type: image/gif$crlf"; print $new_sock "Content-length: 43$crlf$crlf"; print $new_sock $pixel; } shutdown( $new_sock, 2 ); undef($new_sock); } close($sock); exit(0); dbab-1.3.3/bin/dhcp-add-wpad000077500000000000000000000012101357701064400155020ustar00rootroot00000000000000#!/bin/sh ##----------------------------------------------------------------------- ## Porgram: dhcp-add-wpad ## Purpose: Add WPAD (Web Proxy Autodiscovery Protocol) to dbab/dhch service ## Authors: Tong Sun (c) 2014-2019 ## License: BSD license ##----------------------------------------------------------------------- dbabsvr=`cat /etc/dbab/dbab.proxy` dbabsvr_addr=`dig +short $dbabsvr` dbab_conf=/etc/dnsmasq.d/dbab-dnsmasq.wpad.conf # Add wpad DNS entry echo "host-record=wpad,$dbabsvr_addr" > $dbab_conf # Add wpad location to dhcp server echo 'dhcp-option=252,"http://wpad/wpad.dat"' >> $dbab_conf # Report echo File $dbab_conf updated. dbab-1.3.3/dbab.md000066400000000000000000000427411357701064400136370ustar00rootroot00000000000000# Dbab From Start To Finish The following introduction is for `dbab` at or over [version 1.3.1](https://github.com/suntong/dbab/releases/tag/v1.3.1), which is incompatible with previous version (1.2.x) as the configuration files have been renamed. The latest version of this article is [available here](https://github.com/suntong/dbab/blob/master/src/dbab.md). ## Dbab Advantages Before dipping into the following details, here are the advantages of using `dbab` (Dnsmasq-Based Ad-Blocking). First of all, *why this is the best method for ad blocking?* Because all other filter based solutions (privoxy, Adblock-Plus, etc) are CPU intensive because of a large quantity of ad urls and page contents need to be pattern matched, and using regular expressions matching is expensive. Adblock Plus, the easiest choice, is actually the worst choice because it is JavaScript based, and it is the slowest. Furthermore, all these method will more or less alter the rendered web page, to remove the ads. This will be even slower, and might cause side effects as well. The `dbab` is however, using an entirely different approach for ad blocking. It's advantages are: - **Work at the DNS level**. Leave the web pages intact, without any pattern matching, string substitution, and/or html elements replacing. - **Work for your mobile devices as well**. Were you previously in the dilemma of choosing ads free or slow response for your mobile devices (iphone, ipad, etc)? Now you don't. You don't need to install any thing to your mobile devices for them to enjoy the ad-free browsing experience. Moreover, their browsing speed will increase dramatically on revisited pages/images. - **Serve instantly**. All ads will be replaced by a `1x1` pixel gif image locally served by the Pixelserv server. - **Maintenance free**. You don't need to maintain the list of ad sites yourself. The block list can be downloaded from pgl.yoyo.org periodically. If you don't like some of the entries there, you can define your local tweaking that filters them out. - **Easily customized**. It's trivial to add your own entries to the ad blocking list if the existing ones are not enough for you. ## Static IP Now let's start. First, if you haven't done [switching from dynamic IP to static IP](http://sfxpt.wordpress.com/2014/05/11/use-dbab-under-ubuntu-14-04-trusty/) yet, check it out first for how to - configure the static IP, and - add a second static IP address and check out [why to do that](http://sfxpt.wordpress.com/2011/02/21/the-best-ad-blocking-method/#Pixelserv_server_IP_address) as well if you want. Here is a recap what I did to configure my machine with the `192.168.2.102` static IP and a second one of `192.168.2.101`: ```bash cat << EOF > /etc/network/interfaces # interfaces(5) file used by ifup(8) and ifdown(8) # Include files from /etc/network/interfaces.d: source-directory /etc/network/interfaces.d # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 allow-hotplug eth0 # Use static IP instead of dhcp iface eth0 inet static address 192.168.2.102 network 192.168.2.0 netmask 255.255.255.0 broadcast 192.168.2.255 gateway 192.168.2.1 # add a 2nd ip address post-up ip addr add dev eth0 192.168.2.101/24 pre-down ip addr del dev eth0 192.168.2.101/24 EOF /etc/init.d/network-manager restart /etc/init.d/networking restart % ip addr show dev eth0 ... inet 192.168.2.102/24 brd 192.168.2.255 scope global eth0 ... inet 192.168.2.101/24 scope global secondary eth0 ... ``` For details and troubleshooting, refer back to the above [switching from dynamic IP to static IP](http://sfxpt.wordpress.com/2014/05/11/use-dbab-under-ubuntu-14-04-trusty/) document. ## The Plan Once we have our second IP address, the reset of the steps are: 0. [Install & configure DNSmasq](http://sfxpt.wordpress.com/2013/11/30/dnsmasq-installation-configuration-5/). 0. Remove all existing ad blocking tools if you have any. 0. Stop your local web server temporarily if you have any. 0. Before installation `dbab`, go and visit some websites which have ads on their pages such as "yahoo", "abcnews" or anything, then 0. Install & configure the `dbab` package. 0. Restart your local web server if you have any. 0. Now, visit those pages again in different tabs to see if the ads are removed :-). 0. Install [`squid`](http://en.wikipedia.org/wiki/Squid_(software)) caching server, nothing unusual about that. 0. [Setup](https://sfxpt.wordpress.com/2015/11/21/use-new-dbab-to-set-proxy-automatically-3/) [auto proxy](http://sfxpt.wordpress.com/2014/11/23/the-secret-behind-the-auto-proxy-setting/) for everyone and every tool to use the ads-blocking web caching server. That shall be it. Mission accomplished. Details to follow. But please be warned, as there are so many pieces tied together, and thus so many things to configure, the following steps are long. So be warned and be prepared. ## Install & Configure DNSmasq and Dbab To install DNSmasq and Dbab ``` % apt-get update % apt-get install dnsmasq % apt-get install dbab ``` ### Configure DNSmasq To configure DNSmasq: cp /usr/share/doc/dbab/dbab-dnsmasq.service.conf /etc/dnsmasq.d cp /usr/share/doc/dbab/dbab-dnsmasq.intranet.conf /etc/dnsmasq.d The `dbab-dnsmasq.service.conf` provides basic `dnsmasq` service configuration. It's content is pretty standard and consistent across all installations, so you don't need to make any changes to it. The `dbab-dnsmasq.intranet.conf` however, reflects how exactly your intranet is configured. What provided is just a boilerplate, of which every content should be customized. I.e., from the below listing, we can see that the ISP DNS server address, the dhcp lease range, the local-net domain name and the dhcp hosts should all be customized. Edit `/etc/dnsmasq.d/dbab-dnsmasq.intranet.conf` to reflect your true intranet configuration. # == DNS from ISP server=192.168.2.1 # == Dhcp lease (start,end,leasetime) # supply the range of addresses available for lease and optionally # a lease time. If you have more than one network, you will need to # repeat this for each network on which you want to supply DHCP # service. dhcp-range=192.168.2.1,192.168.2.80,48h # == Domain for dnsmasq. this is optional, but if it is set, it # does the following things. # 1) Allows DHCP hosts to have fully qualified domain names, as long # as the domain part matches this setting. # 2) Sets the "domain" DHCP option thereby potentially setting the # domain of all systems configured by DHCP # 3) Provides the domain part for "expand-hosts" domain=EXAMPLE.ORG # == Dhcp hosts. # dhcp-host=00:28:58:3A:EB:A1,192.168.2.20,computer2,infinite # ^ ^ ^ ^ # MAC IP Address hostname lease time # E.g., #dhcp-host=00:16:3e:00:00:01,192.168.0.81,kvm1,8h #dhcp-host=00:16:3e:00:00:02,192.168.0.82,kvm2,8h Now test it: ```bash dig @192.168.2.1 google.ca /etc/init.d/dnsmasq restart dig @192.168.2.102 google.ca echo 'nameserver 192.168.2.102' > /etc/resolv.conf dig google.ca ``` This is heavily-simplified version. For details and troubleshooting refer to: - [Providing DHCP and DNS services with DNSMasq](http://sfxpt.wordpress.com/2011/02/06/providing-dhcp-and-dns-services-with-dnsmasq/) - [DNSmasq Installation & Configuration](http://sfxpt.wordpress.com/2013/11/30/dnsmasq-installation-configuration-5/) #### Faq: dnsmasq: setting capabilities failed If for any reason that you test `dbab` under docker and you get the following error when starting `dnsmasq` (say with `service dnsmasq start`): % service dnsmasq start [....] Starting DNS forwarder and DHCP server: dnsmasq dnsmasq: setting capabilities failed: Operation not permitted failed! The fix is to tell dnsmasq to run as root by adding `user=root` to `/etc/dnsmasq.conf`: ```bash cp /etc/dnsmasq.conf /tmp sed -i '/^#user=/s/$/\nuser=root/' /etc/dnsmasq.conf diff -wU1 /tmp/dnsmasq.conf /etc/dnsmasq.conf # then service dnsmasq start ``` Ref: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=514214 ### Configure Dbab The Dbab comes with a simple local configure using the [second static IP address](http://sfxpt.wordpress.com/2014/05/11/use-dbab-under-ubuntu-14-04-trusty/). To configure `dbab` to work with a local web server: 0. Stop `dbab-svr` service 0. Change the IP address that dbab uses to the second IP address 0. Start `dbab-svr` service 0. Start your local web server again if you have any. You may need to limit its listening port from `0.0.0.0` to your first static IP as well if necessary. In details, do the following as root, again assuming that the server's own IP address is `192.168.2.102`, and its second IP is `192.168.2.101`. The second IP will be used for the `dbab` service (WPAD & pixelserv). ```bash # (run the following as root) # stop dbab service /etc/init.d/dbab stop # use the second IP for dbab-svr to listens on ip -f inet addr show eth0 | awk '/inet /{print $2}' | sed 's|/.*$||; 1d' | sudo tee /etc/dbab/dbab.addr # verify its content before moving on cat /etc/dbab/dbab.addr # if it is not what you intent it to be, correct it with your text editor # or, set it manually (with a different IP address) echo 192.168.2.101 | sudo tee /etc/dbab/dbab.addr # update ad blocking list with the second IP address /usr/sbin/dbab-get-list /usr/sbin/dbab-add-list # OPTIONAL! do the following only if you have squid caching server # and you want to enable automatic WPAD service hostname | tee /etc/dbab/dbab.proxy # NB, if your squid caching server is on a different server, do this instead echo my_squid_server_name | tee /etc/dbab/dbab.proxy # then, /usr/sbin/dhcp-add-wpad # Again verify everything here before moving on because script might not be # 100% time correct. Manually tweaking is inevitable sometimes. # restart DNS & DHCP /etc/init.d/dnsmasq restart # re-start dbab service /etc/init.d/dbab start # re-start your local web server again if you have any # optional, only when dbab will not auto start on boot up update-rc.d dbab defaults ``` That's it. We're done. #### Faq: How to blacklist those bad sites? All these started because there were one time that the top of google hits are often crammed with rubbish sites. I.e., those sites that contains nothing but key words merely to be listed on top of google hits. These sites are called content-farming sites, and goolge has been constantly fighting with them (Google's Farmer Update at the end of February, 2011): > "So-called content farms such as Demand Media and Associated Content, both routinely vilified for churning out shabbily produced, keyword-loaded content that often secured top listings at Google, were penalized severely." [1] [1] http://www.websitemagazine.com/content/blogs/posts/pages/crop-devastation-google-s-farmer-update-retools-rankings.aspx Yet, there are still many content-farming sites that fall through the crack or revamp again. I was so annoyed that, instead of waiting for google to deal with them, I took the matter into my own hand. Here is the updated version that makes use the `dbab` package to block them: First, gather a list of those rubish sites: cat >> /etc/dbab/dbab.list+ The result will look something like this: $ cat /etc/dbab/dbab.list+ dl4all.com filestube.com terapdf.com 101com.com Then, convert the list to be used by DNSmasq: /usr/sbin/dbab-add-list Those bad sites are now blocked by DNSmasq, after restarting it: /etc/init.d/dnsmasq restart That's it. Next time if you accidentally click into those sites, You will see a blank page, which loads instantly, with the following as the page title: (GIF Image, 1x1 pixels) Then you know you've stumbled into sites that you should have avoided. #### Faq: How to whitelist some sites? First see what exactly was listed in the pgl.yoyo.org list. E.g., to enable `www.googleadservices.com`, merely putting `www.googleadservices.com` into `etc/dbab/dbab.list-` won't help, because: $ grep googleadservices /etc/dnsmasq.d/dbab.* address=/googleadservices.com/127.0.0.1 I.e., we should put in `googleadservices.com` instead of `www.googleadservices.com`. Now suppose we need to whitelist `googleadservices.com` and `urlcash.net`, here is how to do: echo 'googleadservices.com' > /etc/dbab/dbab.list- echo 'urlcash.net' >> /etc/dbab/dbab.list- /usr/sbin/dbab-get-list grep googleadservices /etc/dnsmasq.d/dbab.* service dnsmasq restart dig www.googleadservices.com It should show real IP instead of `127.0.0.1`. ## Switching Over to DNSmasq Service To make the above changed configuration take effect, `dnsmasq` must be restarted (because sending SIGHUP to the dnsmasq process will only cause it to empty its cache and then re-load /etc/hosts and /etc/resolv.conf): /etc/init.d/dnsmasq restart But before doing that, we need to disable (DSL) router's dhcp and dns services, because (DSL) router would normally act as both dhchp and dns server for the most cases. if I dedicate a dnsmasq server for both dhcp and dns servers, I have to disable DHCP on my router so only my own dnsmasq server responds to DHCP requests. For DNS, the DHCP response can give the IP address of the DNS for the clients to use. Having restarted the `dnsmasq` service, we still can't test anything about DNS leases because `dnsmasq` doesn't return results for dns query until after it has actually served out the address. So we can't prove anything until after a DHCP/DNS request is made. Again, for details and troubleshooting refer to: - [The Best Ad Blocking Method](http://sfxpt.wordpress.com/2011/02/21/the-best-ad-blocking-method/) - [The Best Ad Blocking Method in a Package](http://sfxpt.wordpress.com/2014/01/05/the-best-ad-blocking-method-in-a-package/) - [Use dbab under Ubuntu 14.04 Trusty](http://sfxpt.wordpress.com/2014/05/11/use-dbab-under-ubuntu-14-04-trusty/) ## Local Caching Server Now it is time to make it easy for anyone visiting your home to enjoy you fast local `squid` caching server. Let's continue on that trend to [auto proxy setting](http://sfxpt.wordpress.com/2014/11/23/the-secret-behind-the-auto-proxy-setting/). I.e., [DNSMasq gets DHCP and DNS together](http://sfxpt.wordpress.com/2011/02/06/providing-dhcp-and-dns-services-with-dnsmasq/), and [the dbab](http://sfxpt.wordpress.com/2014/01/05/the-best-ad-blocking-method-in-a-package/) brings them both and [ad blocking](http://sfxpt.wordpress.com/2011/02/21/the-best-ad-blocking-method/) together, and now let's move a step further to bring [`squid`](http://en.wikipedia.org/wiki/Squid_(software)) and [auto proxy setting](http://sfxpt.wordpress.com/2014/11/23/the-secret-behind-the-auto-proxy-setting/) into the picture and into the harmony. ### Strategy To recap, we need a dedicated server in the SOHO environment for - DHCP, & DNS using [DNSmasq](http://sfxpt.wordpress.com/2013/11/30/dnsmasq-installation-configuration-5/), which we have just installed. - a caching server/proxy using `squid`, which we will install next. - [use `dbab` to provide WPAD & pixelserv service](https://sfxpt.wordpress.com/2015/11/21/use-new-dbab-to-set-proxy-automatically-3/) and join them all together All of them are hosted on a single machine. This is a typical and reasonable configuration, because even with all above, the machine does not need to be a powerful or even a fast one. Mine is a Pentium 5, with 4G of RAM and 300G of disk space, and have a web server, a time server, a printing server, an email server and a SSH server installed as well, along with the DHCP, DNS & web caching server, and it has more than enough power to handle everything. So install the `squid` caching server on this dedicated SOHO server as normal, and start it. The `dbab` should have already properly configured to use it. See above listing for how *"to enable automatic WPAD service"*. ## Verify To check ad blocking, revisit in new tabs those pages you just visited that's full of ads, and compare the differences, or check out the following urls, which are automatically blocked by the `dbab-get-list` command: http://actualdeals.com/ http://ad.about.com/ http://ad.abcnews.com/ http://ad.abcnews.com/anything/else To check your automatic proxy setting, use: $ curl http://wpad/wpad.dat function FindProxyForURL(url, host) { return "PROXY mysohosvr:3128; DIRECT"; } The `http://wpad/wpad.dat` will always be the same regardless how your servers are called, but `mysohosvr` shall be the real name of your squid caching server. To check your automatic proxy results, first [set up your browser to use WPAD](http://goo.gl/9uofLX#heading=h.7wr0f68pdads), then on your SOHO server do the following before visiting any pages: tail -f /var/log/squid3/access.log If the places you are visiting show up in the access log, then everything is working. Now fire up your iphone or ipad to visit some sites. As long as your iphone or ipad is using WIFI from your SOHO network, their visit will be cached as well. Or at least so I read. Check the access log to verify. As for Android, sorry, while iphone or ipad are playing by the rules to set proxy automatically from WPAD, Android isn't. You have to set its proxy manually. Visit some pages with some very-slow-loading pictures, and they visit them again, the picture loading speed will be dramatically faster, especially if your wireless device is not super fast (like mine). If AOK, you may want to setup a cron job to update the block list on a weekly/monthly basis. E.g.: ln -s /usr/sbin/dbab-get-list /etc/cron.monthly/