[Alcatel logo]

AIND DNS configuration

Created 14-Oct-2004
$Revision: 1.7 $ $Date: 2005/09/27 06:36:09 $
Modified  
Mark Owens

Abstract

The DNS is probably the core service required for operation of internets and specifically the Internet. The proper configuration and maintainance is crucial for network operation. While DNS is not really rocket science one slip can cause interesting results. The DNS configuration of AIND came into exsistance during the Xylan days and as such we not only have a delegation from alcatel.com, we also carry several of our own domains (both internally and externally, on the Internet). This document will attempt to define our DNS configuration (both from a conceptual and an operational standpoint) and I will define some best practices that should be employed.


Concepts

We maintain a split-horizon DNS structure. This means that there are two sets of servers for the same zone(s) that project different views of our domain(s) to the internal and external (Internet) networks. This setup has the advantage of being able to expose only those object names to the Internet that external users need to see. A split-horizon configuration can be setup using a variety of methods: BIND8/9 directly support views, or you can use completely seperate servers with different data sets. We are using method 2. This means that we have two primary DNS servers: one with a complete set of internal data, serving internal users; and one with a limited data set, serving the Internet.

The Internet servers are located in the DMZ running on hardened boxes that have no conectivity to the internal DNS structure. There is one primary and four secondaries. While 2 of the 4 secondaries are on the same subnet as the primary, the other 2 are located on different networks (and even connected to different ISPs). This provides a mimnimum level of redundancy and fault protection. The External zone records indeed have all 5 servers listed; though, the first 3 are on the primary Internet connection because it is the fattest

The internal DNS services are supplied by a substantial network of secondary and forwarding servers. We have 5 delegated secondary servers, with at least 12 forwarding servers that user clients can connect to. In addition, each mail handling server (be it gateway, relay, hub, or end item mailbox server) has a fully functioning DNS server that is dedicated to name resolution on that server. In general the only servers that are allowed to request DNS service from the internal primary are the 5 delegated secondary servers and the mail servers. Data center servers are directed to one (or more) of the delegated secondary servers, while end-user workstations are directed to one (or more) of the forwarding servers. In NO situation is a DNS client allowed to connect directly to an Internet DNS server or to pull from the root servers. Indeed the only DNS servers that have that ability are the 5 delegated secondaries.

The ind.alcatel.com delegation comes from the Alcatel corporate internal servers, which delegate several other internal domains. In order to maintain connectivity to these internal domains, we have a copy of the internal primary servers, including the delegated masters. This map is maintained in a global named.boot file located at fwadmin.alcatel.com. The file is updated periodically and we maintain a manual sync. That is to say someone must periodically grab the global file and mash it into our systems. The file is in the old bind version 4 format, so is is manipulated using this procedure.

 

Figure 1: Internal DNS block diagram. Internal DNS Block Diagram - click to enlarge
(click to enlarge)

 

The following table lists the principle DNS servers that present the internal view of our zones. The servers are a combination of fully delegated slaves, to forwarders and some ghost slaves (ie slaves that hold complete zone data but are not listed as as delegated servers - not NS records for them). This table also lists the servers for all of ind.alcatel.com sub-domains.

 

Table 1: Internal DNS Servers 22-Nov-2004
Server Name IPv4 Address Type Bind version Zone(s) Host Info Comments
nameserver 198.206.181.139 Master ISC 9.3.0+SSL0.9.7e+validator.c ind.alcatel.com
xylan.com
ALL
Sun Fire V100
Sol9
All server configuration files are located here.
mailhub1 198.206.181.170 Slave ISC 9.3.0+SSL0.9.7e+validator.c (see nameserver) Ultra E420
Sol9
mailhub2 198.206.181.70 Slave ISC 9.3.0+SSL0.9.7e+validator.c (see nameserver) Ultra E420
Sol9
data 198.206.187.58 Slave ISC 9.3.0+SSL0.9.7e+validator.c (see nameserver) Ultra E450
Sol2.7
trunks 198.206.187.57 Slave ISC 9.3.0+SSL0.9.7e+validator.c (see nameserver) Ultra E420
Sol2.7
booty 198.206.181.22
198.206.186.31
198.206.187.5
208.8.1.7
128.251.24.198
Slave ISC 9.3.0+SSL0.9.7e+validator.c (see nameserver) Ultra E250
Sol2.7
tempis 198.206.181.123
10.251.9.123
192.168.9.123
Forwarder
Master
ISC 9.3.0+SSL0.9.7e+validator.c ntp.ind.alcatel.com Ultra 1/170E
Sol9
Stratum 0 Radio clock
redford 198.206.181.85
10.255.10.80
Ghost/Forwarder
Master
ISC 9.3.0+SSL0.9.7d test.ind.alcatel.com Ultra E250
Sol9
Master for test net domains.
carbon 198.206.181.155
10.255.10.4
128.251.13.35
192.168.117.35
Forwarder
Slave
ISC 9.3.0+SSL0.9.7d test.ind.alcatel.com Ultra E450
Sol9
BIND drops requests from the 128 and 192 i/fs
testors 198.206.181.51
10.255.10.2
Forwarder
Slave
ISC 9.1.0 test.ind.alcatel.com Ultra E450
Sol2.5.1
Needs Upgrade
listproc 198.206.181.113 Local Forwarder ISC 9.3.0+SSL0.9.7e+validator.c N/A Ultra E250
Sol9
local sendmail resolver
omni 198.206.181.20 Local Ghost ISC 9.3.0+SSL0.9.7d N/A Ultra E250
Sol2.7
local sendmail resolver
newman 198.206.187.6 Local Ghost ISC 9.1.0 N/A Ultra E420
Sol2.6
local sendmail resolver
mrelay1 198.206.181.172 Local Ghost ISC 9.3.0+SSL0.9.7d N/A Netra T1/105
Sol9
local sendmail resolver
mrelay2 198.206.181.177 Local Ghost ISC 9.3.0+SSL0.9.7d N/A Netra T1/105
Sol9
local sendmail resolver
odm1svcs 128.251.13.33 Forwarder
Master
ISC 9.3.0+SSL0.9.7d odm1.ind.alcatel.com Netra T1/105
Sol9
ODM1 (Wipro) development sandbox 1
sustsvcs 128.251.13.161 Forwarder
Master
Debian 9.2.4-1 sust.ind.alcatel.com Rackable Systems 2xPIII
Linux 2.4.27-2-686-smp (sarge)
ODM1 (Wipro) sustaining sandbox 1
odm3svcs 192.168.117.33 Forwarder
Master
RH 9.2.2-P3 odm3.ind.alcatel.com Rackable Systems 2xPIII
Linux 2.4.22 smp (fc1)
ODM3 (HSS) development sandbox 1
ab1 198.206.187.11 Workstation Forwarder ISC 9.3.0+SSL0.9.7d Netra X1
Sol8
Desktop resolver
ab2 198.206.186.82 Workstation Forwarder ISC 9.3.0+SSL0.9.7d Netra X1
Sol8
Desktop resolver
ab3 198.206.181.198 Workstation Forwarder ISC 9.3.0+SSL0.9.7d Netra X1
Sol9
Desktop resolver
osprey 198.206.181.26 Server Forwarder ISC 9.3.0+SSL0.9.7d Ultra E250
Sol9
Server resolver
arusha 198.206.181.32 Server Forwarder ISC 9.3.0+SSL0.9.7d Ultra E250
Sol9
Server resolver
backup 198.206.187.139 Server Forwarder ISC 9.3.0+SSL0.9.7d Ultra E450
Sol2.7
Server resolver
debcomp 198.206.181.94 Server Forwarder Debian 9.2.4-1 Sun V60
Linux 2.4.27-1-686 smp (sarge)
Server resolver
derby 198.206.187.139 Server Forwarder RH 9.2.1 HP DL320
Linux 2.4.18-14 (RH9)
Server resolver
srvcgate 208.8.0.245 Ghost Slave ISC 9.3.0+SSL0.9.7e+validator.c (see nameserver) Ultra E420
Sol9
DMZ services name resolution
dmzns 208.8.0.247 Ghost Slave ISC 9.3.0+SSL0.9.7e+validator.c (see nameserver) Netra X1
Sol9
DMZ services name resolution

 

The external Internet view is maintained by 5 servers. 2 of these are located offsite (additional backup may at some time be performed by Alcanet). As stated earlier, the servers are single function boxes running either Linux, or Solaris, that have been hardened. The BIND software is running as a non-root user, in a protected chroot jail, located in /opt/jails/bind.

Additional security measures have been configured on the servers. External requests are assumed to be only for the authoratative data hosted by these DNS servers; therefore, forwarding, recursion, and glue-fetching are turned off. This prevents our external from being used as forwarding servers and keeps the cache at 0. (these are known as delegated nameservers.) Zone transfers are limited to our secondary servers only. The CHAOS class version.bind. and authors.bind. information has been modified, to hamper version specific attacks.

 

Figure 2: External DNS block diagram. External DNS Block Diagram - click to enlarge
(click to enlarge)

 

Table 2: External DNS Servers 14-Oct-2004
Server Name IPv4 Address IPv6 Address Type Bind version Zone(s) Host Info Comments
ns1 208.8.0.241 fe80::a00:20ff:fec2:75a6/10 Master ISC 9.3.0+SSL0.9.7d ind.alcatel.com
xylan.com
ALL Vanity
Netra T1/105
Sol9
primary external nameserver
mailgate1 208.8.0.225 ::1/128 (unconfig) Slave ISC 9.3.0+SSL0.9.7d (see ns1) Netra T1/105
Sol9
mailgate2 208.8.0.226 ::1/128 (unconfig) Slave ISC 9.3.0+SSL0.9.7d (see ns1) Netra T1/105
Sol9
ns.aindsand.net 67.116.113.190 N/A Ghost Slave ISC 9.3.0+SSL0.9.7d (see ns1) Ultra 5
Sol9
BIND9 integrated split-horizon setup
motech.net N/A N/A Ghost Slave ISC 9.1.2 (sun) (see ns1) Netra T1/105
Sol8
BIND9 integrated split-horizon setup

 


Operational

We currently maintain 12 internal DNS domains. Two upper level: xylan.com and ind.alcatel.com; and 9 sub-domains of ind.alcatel.com. The following table lists the specific domain information. For information on the external domain information, please see the Alcatel IND Registered Names and Numbers document.

We are currently running BIND version 9 on most of the servers. The package is compiled on a Sun Ultra 60 running Solaris 2.7. This allows the package to be used on Solaris 2.7 and above. Obtain the ISCbind9 package (SysV pkgadd format). If you will be running BIND on Solaris 2.6 (or earlier), you could pull BIND8 or BIND9 from testors:/opt/bind/bin-8.2.4 or testors:/opt/bind/bin-9.1.2. (tho you'd be better off upgrading!)

 

Table 3: Internal Domain Details 14-Oct-2004
Domain Name Primary Server Contactr Comments
xylan.com nameserver hostmaster@ind.alcatel.com
ind.alcatel.com nameserver hostmaster@ind.alcatel.com Migration to DNSCES/TSIG
utah.ind.alcatel.com crypto Jeff.Hammett@ind.alcatel.com Utah R/D
salem.ind.alcatel.com nova Wayne.Nakata@ind.alcatel.com Salem R/D
test.ind.alcatel.com redford hostmaster@ind.alcatel.com TestNet
dhcp.ind.alcatel.com nameserver hostmaster@ind.alcatel.com
nms.ind.alcatel.com ??? Ron.Traver@ind.alcatel.com
aind.ind.alcatel.com ns1.aind.alcatel.com Lance.Choi@ind.alcatel.com MicroSoft AD
odm1.ind.alcatel.com odm1svcs hostmaster@ind.alcatel.com Wipro XOS jail
odm3.ind.alcatel.com odm3svcs hostmaster@ind.alcatel.com HSS AOS-reuse jail
sust.ind.alcatel.com sustsvcs hostmaster@ind.alcatel.com WIPRO Sustaining jail
ntp.ind.alcatel.com tempis owens@ind.alcatel.com NTP cluster (and support stuff)
acton.ind.alcatel.com ??? ???

 

DNS zone data and configuration files are currently maintained in two locations: the internal master server, nameserver; the external master server, ns. (At some point in time the external zone data may be sourced from nameserver onto ns... but that day is NOT here).

The primary internal configuration data is located on nameserver:/opt/bind/.... The configuration files are broken into several include files, sourced from subdirectories under /opt/bind. The primary named configuration file named.conf is a symbolic link to the actual configuration file located in the /opt/bind/conf directory. This file includes common named operational definitions located in files within the /opt/bind/conf directory. Zone definitions are also found in this directory. The actual zone data is stored in the following directories (depending on the type of data): master, legacy, vanity, in-addr, rfc2317, slave, stubs. Note: the etc, staging, and work directories are for internal maintainance use only.

 

Figure 3: Database file structure.
      /opt/bind \
                |
                +- conf \
                |        +- acls.conf (shared ACLs)
                |        +- bogon.conf (bogus networks)
                |        +- ndc.conf (ndc data)
                |        +- server.defs (server data)
                |        +- server.local (control channel data)
                |        +- dnssec.keys (server TLS data)
                |        +- conf.zone-type (zone definitions)
                |
                +- master \
                |          +- local-cache (ALCATEL cache)
                |          +- named.cname-hosts (Common CNAME data)
                |          +- named.ind-delegations (Common sub-delegation data)
                |          +- named.bind (CHAOS class data)
                |          +- named.local (127.0.0 zone data)
                |          +- named.ns-records (Common NS data)
                |          +- named.root (ROOT cache)
                |          +- named.net-number (Common object data)
                |          +- zone-name (IND zone SOA data)
                |
                +- legacy \
                |          +- zone-name (Legacy domain zone data)
                |
                +- vanity \
                |          +- zone-name (Vanity domain zone data)
                |
                +- in-addr \
                |           +- named.net-number (in-addr zone data)
                |
                +- rfc2317 \
                |           +- named.net-number.rev (RFC2317 PTR data)
                |
                +- slave \
                |         +- named.net-number.rev (IND slave xfer in-addr data)
                |         +- zone-name (IND slave xfer forward zone data)
                |
                +- stubs \
                |         +- db.rev.net-number (ALCANET slave xfer reverse zone data)
                |         +- db.zone-name (ALCANET slave xfer forward zone data)
          
  

 

The forward definitions for the ind zones are composed of multiple files: the zone SOA data; the NS records; the sub-delegation data; and one (or more) object detail files. The zone SOA data is found in the file named after the DNS domain (e.g. ind.alcatel.com). These zone files then include the following files: named.ns-records; named.ind-delegations; followed by one (or more) occurences of the object detail data files. The object detail data is stored in named.netnum-hosts files. This configuration allows use to share the detailed host data in multiple zones without duplication of the zone files. As usual, you must update the zone serial number when any changes are made to the zone. This includes changes to the named.ns-records or named.ind-delegations files.

The reverse zone (in-addr.arpa) data for each sub-net is completely defined in one file (the NS data is included from ../master/named.ns-records)

The individual vanity and legacy zones are completely defined in seperate files.

General bind configuration file changes and zone additions or deletions must be distributed to all of the slave servers. (these are changes that are not contained in the zone data). The master configuration files are not suitable for direct use by the slaves. When the primary configuration files are modified, the files need to be crafted into formats suitable for the slave and forwarding servers. This function is handled by make (and /opt/bind/Makefile). Essentially make is used to drive the build process that consists of some nawk scripts and a few RCS checkins. Once the source files have been mashed correctly, ssh is used to push them onto specific slave/forwarding servers. The final piece is to restart named on the servers that have been updated. Currently this restart is effected by calling /etc/init.d/bind.server restart via ssh; thus, implying the need for all of the servers to have a copy of root@nameserver's public key.

The primary external configuration data is located on ns.ind.alcatel.com:/opt/jails/bind/opt/bind/{conf,master}. Any modifications needed to the Internet DNS view must be made to files in this location. There are two broad classes of changes that might be made: 1) zone modification, such as adding or deleting objects within a domain (not including delegation of sub-domains); 2) server configuration changes, including adding or deleting of DNS zones. For all simple zone modifications, mearly edit the master zone file(s) and restart BIND (via /opt/init.d/bind.server restart). However; if you are adding or removing DNS zones you must first make the appropriate changes in the configuration files located in /opt/jails/bind/opt/bind/conf; THEN you must make the appropriate configuration changes to the secondary server(s). Once the secondary [server(s)] has been modified you may restart the primary BIND server, and then restart the secondary server(s).

 

Global named.boot manipulation

The global named.boot file is used to define stub zone data for all of the internal Alcatel zones. Unlike an internal root server, where the child delegations would look to the root for all non-child data; each child domain must have glue records that define all of the primary name servers for each child domain. There are two ways to do this: 1) define our master server as a fully functional slave for all domains; or 2) define stub zones for all domains. We use stub zones because these only maintain the NS resource records; thus, we don't transfer all of the zone data for all of the internal domains. In any case we have to copy this stub data into our primary servers periodically. Unfortunatly I have not automated the global named.boot updates in any way. So the following procedure is used to update the primary master nameserver, the build process will format the file and install it onto the slave servers. Additionally this file defines the internal Mail eXchangers so you will need to extract the MX data and update the mailertables on the internal mailhubs (not documented here).

  1. Login to nameserver and cd to /opt/bind/work
  2. mv the in.boot file to in.boot.sav
  3. Login to the fwadmin.alcatel.fr server and copy the file into the working directory on nameserver:/opt/bind/work as in.boot
  4. Within vi perform the following actions:
    Remove the ind and xylan NS records
    Remove the 198.206.18[1-7] in-addr NS records
    Swap secondary with stub :g/secondary/s/secondary/stub/
    Prepend stubs directory to file names :g;db.;s;db.;stubs/db.;
  5. Save the resulting file as infile
  6. Use the following command to convert the file into a bind9 format conf file:
    cat infile | /opt/ISCbind9/libexec/named-bootconf.sh > outfile
  7. Within vi perform the following actions:
    Remove the options stanza at the top
    Copy the RCS $Id: dns.html,v 1.7 2005/09/27 06:36:09 root Exp root $Id: keyword. (this preservers the original Alcanet revision data.)
    Translate hash comments into C++ style via: :g;#;s;#;\/\/;
  8. Save the file as conf.alcatel
  9. Copy the conf.alcatel file to /opt/bind/conf
  10. cd into /opt/bind and run make. This will update the master servers and restart the systems.

 


Physical data

The master BIND server nameserver.ind.alcatel.com is configured for IP Networking Multipathing using the dmfe0 and dmfe1 interfaces. The virtual IP is 198.206.181.139. The physical interface dmfe0 is 198.206.181.41, dmfe1 is 198.206.181.111, these are used as the probe addresses. The failover group is issha. We are not using true IP Multipathing, in that we are only using one VIF in a failover (not load balanced) mode; thus, we only need 3 IP addresses.

The following steps are used to configure the IP failover: (see SunSolve infodoc 3212)

 


SPF/Sender ID and DomainKey publication

The usefulness of email on the Internet has been greatly dimished by UCE, SPAM, and other unwanted/undesired email. There are many ways to reduce this email flow, including: black-listing, white-listing, hueristic filters, etc... Black-listing and white-listing work by creating lists of (dis)allowed senders. Messages are then filtered based on the claimed sender ID. The problem with this type of filtering is that the sender ID can be easily forged and current SMTP implimentations do not have any method to authenticate the claimed sender. (until now)

SPF/SendrId utilize DNS TXT records to define which machines (A, MX, or PTR records) are authorized to source email into the Internet. The assumption here is that you as the owner of the DNS domain are the only one that can publish data pertaining to email source machines. It should be understood that SPF only authenticates which machines are valid sources of email, there is nothing stopping a spammer from publishing SPF records. However, the SPF validates the source as who they say they are (assuming that the source matches the SPF data), with this authentication, you can now impliment a robust black/white list. Additionally you can decide to drop email that fails the authentication.

Our external DNS holds SPF TXT records for each domain that we own. There are two SPF definitions that we publish depending on wether the domain sources email or not. All domains other than the ind.alcatel.com domain do NOT source email; therefore, the SPF data for those domains will be in the form of:

        IN      TXT     "v=spf1 -all"
    
The ind.alcatel.com domain will carry the following SPF data:
        IN      TXT     "v=spf1 ptr a:postal?.ind.alcatel.com ~all"
    

DomainKeys utilize public key cryptography to sign all outbound email. The public key is published in special DNS TXT records, while the private key is, well... held in a secure fashion. Outbound email is signed by the private key prior to egress onto the Internet. The receiving side can now authenticate the sender by using the public key (that is availible from DNS). As is the case with SPF/SenderId, DomainKeys only validate the sender, it is still up to the receiver to do something with the message.

Our external DNS holds DomainKey TXT records for the ind.alcatel.com and vanity domains. The following public keys are published:

    

 


DNSSEC/TSIG and crypto data with DNS zones

The authenticity of DNS data is a growing concern within some groups. Because machine identities are based in part on DNS identification, it is important to provide a method of providing provably authentic DNS data (!?). Additonally a better method than limiting zone transfers via source IP address should be used. Finally, some applications can be configured to use cryptographic data that is placed into the DNS. (OpenSSH and SenderID are two such applications). But....

Before you can rely on the DNS data for these types of things, you must be able to authenticate your DNS data. This is accomplished via DNSSEC and TSIG. DNSSEC uses public-key cryptography to sign, and thus authenticate, DNS zones (and subsequently the data within). TSIG is