Operating one's own local DNS resolution servers is one of the simplest and lowest-cost things an IT administrator can do to monitor and protect applications, services, and users from potential risks.

Paul Vixie, Chairman & CEO, Farsight Security, Inc.

October 24, 2018

9 Min Read

A non-networked computer is very rare now because most useful work involves data and services that are distributed across some kind of "internet" — that is, any network that speaks IP, public or private. Notably, most non-internet networks have failed — we won't be building or connecting to AppleTalk or DECnet or SNA or XNS networks ever again. Consider, then, that most useful work relies on TCP/IP, which itself relies on the Domain Name System (DNS). We should know as much as possible about our use of the DNS, but most IT administrators are much too busy learning their specialized craft to also become experts on every enabling technology.

In this article, I will describe the use of DNS by users and applications, which means DNS resolution. There's much more than this to say about DNS — for example, DNS content management and provision. But while providing DNS content is a specialized task, accessing that DNS content through resolution is a universal activity — no work of any kind can begin in TCP/IP networks until DNS has been accessed at least once and possibly many times. Importantly, for this discussion, among the kinds of "work" I refer to are online crime, network abuse, and surveillance.

Early DNS
In the 1980s, when TCP/IP's design was mostly finalized by the research and education communities, it was common to have only one or a few computers on a campus because computers were very large and expensive to acquire, operate, and maintain. This concentration caused many services and applications to share a single computer, which users would access using "dumb" terminals. Thus, when the DNS was created, the resolution service was installed on at least one computer on every campus. We learned to expect rapid answers to most DNS questions, usually receiving an answer within a millisecond or so. Networked applications such as e-mail or file service were operated under the assumptions that at least one DNS access would be necessary for every "transaction" but that DNS access would be fast enough that this would not reduce the performance of applications and services that depended on it.

Before the commercialization and privatization of the Internet in the 1990s, there was no online crime or abuse. When only scientists and engineers could access the Internet, and the goals were academic rather than commercial, we had safety by accident. Much of today's Internet crime and abuse is made possible by the total absence of security consideration in some of the Internet's oldest core protocols and services — and it has proved very difficult to address security considerations decades later to a network that was designed to accommodate only trusted users and trusted applications. One of the most serious security problems is that malicious actors can create DNS content to facilitate their online crimes, and they will receive the same excellent service for this malicious content as we get for non-malicious content.

Today's DNS
Continuous consolidation among Internet companies has created some dominant players, such as Google, Cisco, IBM, and Cloudflare, each of which offers a global DNS resolution service identical to the resolution service every network had to run for itself back in the early days. This is in some ways a natural progression because these external DNS resolution services (such as 8.8.8.8, 9.9.9.9, or 1.1.1.1) are free, convenient, and, perhaps unknown to most users, also a vital source of network intelligence for the companies that operate these resolution services. Many IT administrators working today were not working in the field when DNS resolution service was always provided on-campus, under local policy, by local staff. As a result of this trend and that generation gap, there are added costs and subtracted benefits, or losses, from the use of external DNS resolution servers, either the global kind (8.8.8.8, et al.) or the regional kind (as provided by one's own ISP).

Accounting of Losses
The first loss we experienced from externalizing the once-local DNS resolution service was privacy. Our external DNS transactions are rarely protected from surveillance, and while such protection is now being developed, that protection will come in the form of added complexity. The best way to avoid having one's DNS transactions observed, tracked, or analyzed by third parties is to not externalize those transactions in the first place. Most of us need not fear surveillance of traffic inside our own networks, which makes those "owned networks" a great place to keep data we don't want published.

The second loss from externalizing DNS resolution is performance. No matter how many DNS resolution servers are externally constructed, none will be reachable by our users and our applications in less than 1 millisecond. Thus, the number of transactions we can process per unit of time will be reduced by the need to wait for speed-of-light transmission delays across a distance greater than our campus. Most web browsers now contain an internal DNS "cache" to offset this penalty. Most other networked applications lack this cache. Again, I question the need for application-level caching when a simple local DNS resolution server would offer the same benefits to all networked applications across an organization.

(Column continues on next page.)

The third loss we experience as a result of using external DNS resolution servers instead of operating such servers locally on our own campus is in monitoring. Most malware and most botnets make use of the DNS to reach their command and control operators, and some use the DNS as a steganographic exfiltration channel for the victim's personal information. Every modern DNS resolution service has some way to monitor "outbound" traffic to detect insider threats. However, to get the benefit of such monitoring, it's necessary to see your own traffic. If every application running inside the network has a direct relationship with an external DNS resolution service, then only the operator of that service — not the organization's IT administrator himself — will be able to monitor that traffic for threats. Some of these resolution service operators offer monitoring to their users, but most do not, and those that do sometimes monetize their observations of that traffic. (This is known as "surveillance capitalism." For example, the DNS queries coming from the customer of an ISP might be data mined for keywords that are then used to send focused advertising content to that customer.)

The fourth loss we can experience from using an external rather than a local DNS resolution service is control. Modern DNS technology includes the capability of policy enforcement, whereby malicious DNS patterns are rejected by the resolution server based on policy settings (from the local security operations center) and policy subscriptions (from external security information providers). Policy of this kind can reject resolutions based on known-bad domain names, name server names, name server addresses, or distant mail/web/content server addresses. However, this incredibly powerful capability is available only to operators of local DNS resolution servers. Some external DNS resolution providers offer this kind of policy filtering but without the fine-grained control that a locally operated server can provide.

Methodology
Despite this growing shift toward external DNS resolution services, it's not necessary to completely choose only one way that DNS resolution is provided to all local users and servers. A mixture of internal and external DNS resolution consumers is not only possible but quite common. The DNS resolution service to be used by a server is usually set in its control panel or in the private cloud's control panel, whereas for a desktop or laptop or mobile device, it is set in the control panel of the DHCP server. So, it's possible to experiment and to experience a very gradual and surprise-free transition from use of external DNS resolution servers to internal ones.

Every open source server platform, such as Linux or BSD, offers many free implementations of the DNS resolution service. The oldest of these is called BIND, but newer implementations such as PowerDNS, Unbound, and Knot are also well-trusted, production-ready software packages. Most will offer some kind of template configuration that includes local DNS resolution. While you certainly will tune and enhance that configuration over time, the "jumping in" cost is extremely low. Commercial products are also available for DNS resolution, from vendors such as Microsoft, Infoblox, Nominum, BlueCat, and others.

Diversity is essential. Every campus needs at least two independent DNS resolution servers, ideally situated on different LAN segments with different power sources. Safe configuration is also essential — operators must ensure that no query coming from outside the network will be answered because this is a well-known and quite popular method for attackers to amplify their distributed denial-of-service capacity. It's worth spending half a day studying documentation, HOWTO files, and forums before turning up any new service — but that's especially true for a DNS resolution service.

The general outline of activities related to establishing and operating a local DNS resolution service on your campus is as follows:

  1. Research and experiment to find a solution and configuration you can live with initially.

  2. Deploy at least two diversified internal servers, which are probably virtual servers.

  3. Configure management to cause software, configurations, and updates to be automated.

  4. Conduct testing, from both on-campus and off-campus, to ensure correct (and only correct) operation.

  5. Monitor, including both operational events like reachability, but also transaction logging.

  6. Reconfigure a small set of test subjects, including some servers and some end users.

  7. Maintain a watchful settling-in period to gain measurable confidence about the new methodology.

  8. Reconfigure a larger set of test subjects or, perhaps, the remainder of the campus.

  9. Deploy a testing server for experimenting with automated logging and policy filtering.

  10. Research and experiment on different alternatives for transaction monitoring.

  11. Research and experiment on different alternatives for resolution policy filtering.

  12. Perform a gradual rollout into production of new solutions that have good results from experimentation.

Conclusion
Operating one's own local DNS resolution servers is one of the simplest and lowest-cost things an IT administrator can do to monitor and protect their applications, services, and users from potential risks. These risks — including surveillance capitalism, unmanageable external dependencies, attacks carried via DNS, and attacks that could be detected via DNS — have a much higher potential cost than the mitigation strategy outlined here. Additionally, the DNS resolution service is so central to every other IT-related activity that any and all IT administrators who take the time to investigate and master this technology will amplify their effectiveness and the value they bring to their enterprise.

Related Content:

 

Black Hat Europe returns to London Dec. 3-6, 2018, with hands-on technical Trainings, cutting-edge Briefings, Arsenal open-source tool demonstrations, top-tier security solutions, and service providers in the Business Hall. Click for information on the conference and to register.

About the Author(s)

Paul Vixie

Chairman & CEO, Farsight Security, Inc.

Dr. Paul Vixie is an Internet pioneer and thought leader who designed, implemented, and deployed several Domain Name System (DNS) protocol extensions and applications that are used throughout the Internet today. He is CEO of Farsight Security Inc. Previously, he served as President, Chairman, and founder of Internet Systems Consortium (ISC); as President of MAPS, PAIX, and MIBH; and as CTO of Abovenet/MFN. Since 2005, Dr. Vixie has served on the ARIN Board of Trustees. He is a founding member of ICANN's Root Server System Advisory Committee and Security and Stability Advisory Committee. He operated ISC's F-Root name server for many years and is a member of Cogent's C-Root team. He is a sys admin for Op Sec Trust. He earned his PhD from Keio University, for work related to DNS and DNSSEC, and was recently named to the 2014 Internet Hall of Fame.

Keep up with the latest cybersecurity threats, newly discovered vulnerabilities, data breach information, and emerging trends. Delivered daily or weekly right to your email inbox.

You May Also Like


More Insights