Download Network Security Via Explicit Consent and more Exercises Computer Networks in PDF only on Docsity!
Network Security Via Explicit Consent
Jad Naous∗, Michael Walfish†, David Mazi`eres∗, Antonio Nicolosi‡, and Arun Seehra†
∗Stanford †UT Austin ‡Stevens Institute of Technology
Abstract
Securing real-world operating systems is hard; drop- ping packets headed to those systems is easy. Thus, network-layer defenses have become indispensable to end-host security. Unfortunately, most defenses inflict collateral damage, require hardware modification, or ne- cessitate coordination between organizations’ adminis- trators. Yet, for all that, each defense addresses only a subset of attacks. This paper describes ICING, a network layer that allows all stakeholders (senders, receivers, and providers) to deploy new network defenses unilaterally, with enough precision to avoid collateral damage, and without further hardware modification. ICING captures many prior network-layer defenses within a coherent framework: for a packet to flow from sender to receiver, every entity along the path must have consented to the entire path. To enforce this property, ICING’s data plane must address a key challenge: how mutually distrustful realms that cannot rely on per-packet or per-flow public key cryptography ensure that packets follow their pur- ported paths. We demonstrate ICING’s technical feasibil- ity with a prototype that forwards at over 2 Gbit/s.
1 Introduction
Although the Internet architecture is commonly regarded as a disaster for end-host security, it does offer some se- curity benefits. The assignment of functions to layers al- lows people to sidestep intractable systems problems by relying on the network as a “choke point”: rather than secure real-world operating systems, people filter pack- ets headed to those systems. But of course Internet security is a disaster. First, the architecture enables attacks (malware epidemics, denial-of-service, route hijacking, etc.). Second, it in- hibits defenses: conceptually simple actions, like iden- tifying and blocking a troublesome traffic transmitter, are research problems. More generally, network-layer defenses—ACLs, VPNs, NATs, firewalls, probabilistic signature matching, pleading with your ISP at midnight, pleading with your attacker’s ISP at midnight, whitelist- ing, blacklisting, redirecting traffic to a DoS defender, securing BGP, network capabilities, proof-of-work, up- grading your router, etc.—inflict collateral damage, re- quire hardware modifications, or necessitate coordina- tion between organizations’ administrators. Worse, each defense addresses only a subset of at- tacks, so people incur the above costs repeatedly. Worse
still, not every defense can coexist with every other. And worst of all, attacks continue to evolve. Is a better world possible? We note that despite the continual evolution of attacks, the network stakeholders—senders, providers, and receivers—remain constant. This paper presents a new network layer, ICING. ICING’s deployment requires hardware modification— but only once. In an ICING network, each stakeholder can deploy new network defenses unilaterally, with enough precision to avoid collateral damage, and without further hardware modification. Many proposed defenses that so far require different, overlapping mechanisms can, under ICING, coexist in a coherent framework.
ICING divides the network into mutually distrustful administrative realms. To communicate with a destina- tion, the sender must identify an appropriate sequence of realms, which we call a path, between it and the destina- tion. ICING’s approach to “path finding” involves senders contacting path servers that are akin to today’s DNS servers (an approach inspired by [43]), but ICING also works with any other approach to path finding. Before sending a packet along a path, a sender must get explicit consent from each realm. (For now, we take the two endpoints to be their own realms, but we revisit that simplification in §3.4.) To get a realm’s consent, a sender communicates with a general-purpose server that is physically separate from the realm’s forwarding hardware (a decomposition inspired by [14, 15, 23, 37]). The sender proposes the path. In making its decision, the server can incorporate arbitrary factors besides the proposed path (billing relationships [36], authentication, etc.). Upon consent, the server issues a proof-of-consent (PoC) that authenticates the proposed path. PoCs gener- alize network capabilities [8, 36, 42, 44] and Visas [18] (see §8). On the data path, an honest forwarder drops packets with invalid PoCs; it also drops packets that have deviated from their authorized paths. In practice, many “control plane” steps can be com- bined; for example, the sender may, in one step, obtain a path and PoCs for a large subsequence of that path.
ICING upholds a coherent security property. We say that the path, P, taken by a packet is conforming when P itself conforms to the security policies of the enti- ties along P—the sender, the receiver, and all transited realms. The process described above ensures that pack- ets flow from senders to receivers only along conforming
paths, a property that we call consent-to-connectivity. Consent-to-connectivity subsumes or generalizes the high-level goals of many prior proposals (§7.3). In those proposals, some stakeholder along a packet’s path decides whether to drop the packet. The decision is based on various factors, but one frequent factor is the path itself, or rather, some component of the path— the source [6, 8, 13, 25, 32, 42, 44], the two end- points [18, 24, 36], a suffix [41], a prefix [9], an arbitrary subsequence [22], the whole path [34], etc. A network that lets entities grant consent on an arbitrary basis in the control plane and that enforces consent-to-connectivity in the data plane serves to unify these proposals. Consent-to-connectivity also leads to security innova- tions (§7.3). For example, under ICING, receivers and providers can control paths toward them. Such control allows them to invoke network services selectively (e.g., forcing the traffic of certain sources or classes to go through a third-party deep packet inspector). In explicitly empowering intermediate providers to reject flows, consent-to-connectivity apparently violates the IPv4 ideal of global connectivity. But we observe that today providers can and do drop packets with no re- course for the endpoints. Under ICING, providers can ex- ercise their legitimate interests but can do so explicitly, while endpoints can choose any conforming path. And, because decisions are made in general-purpose servers, prior to packet flow, they can incorporate arbitrary evi- dence, permitting them to avoid the coarse-grained rules (e.g., “block all traffic on particular ports from particular areas of the network”) that cause collateral damage. This separation of control plane software from for- warding hardware facilitates innovation, as articulated by others [14, 15, 23]. ICING’s contribution to this separa- tion is to permit a realm to unilaterally express security policies in its own software but to have other realms’ forwarding hardware participate in enforcing those poli- cies (§3.4,§7). More generally, deploying new security measures under ICING is closer to updating one’s DNS servers than one’s routers (§7.1). And, ICING encourages other kinds of innovation; for example, it permits coex- isting inter-domain routing protocols (§7.1). Our main focus in this paper is on upholding consent- to-connectivity, which requires addressing some chal- lenges (§3). First, it is not clear how a sequence, S, of mu- tually distrustful realms can, without prior coordination or prohibitive per-packet public key cryptography, ensure that a packet follows S only if the realms along S con- sented to S (§3.2). Second, to avoid the overhead of ex- ecuting the consent-granting process for every flow that it carries, a realm needs a way to selectively delegate its authority (§3.4). Third, senders must request consent, but how do they get consent to request the consent? In prior work, such bootstrapping requires great care [10, 32, 44];
ICING must address a similar challenge (§3.5). ICING is technically viable: our prototype forwarder, built on an FPGA (§5), forwards at over 2 Gbit/s for all packet sizes. A production ASIC implementation of ICING would of course be far faster and could handle backbone links. We believe that ICING’s space overhead is heavy but acceptable: for 1514-byte packets, the aver- age header is 18% of the packet; for IP it is 1% (§6). However, technical viability alone does not address the questions of how to gain adoption and how ICING coex- ists with other proposals of similar scope. We make only passing reference to these questions (§8, §9) and for now note that ICING promotes the coexistence of naming, routing and access control mechanisms. Ultimately, we believe that these questions are premature. Our purpose in this paper is to give an existence proof that ICING’s goals are realizable; that proof must precede discussion about backwards compatibility with the Internet or co- habitability with other proposals, because without know- ing if something is possible, we certainly cannot figure out how to deploy it. Many other “clean slate” architectures have been pro- posed recently, some of which partially inspire this work (§8). ICING’s contributions are its overall architec- ture (§2), particularly that mutually distrustful realms co- operate to enforce each other’s policies; the articulation of consent-to-connectivity as a unifying property (§7); a protocol that upholds this property (§3); and a prototype that serves as an existence proof (§5,§6).
2 Overview of ICING and rationale We now give an overview of ICING and describe the ra- tionale for our high-level approach. §3–§4 detail ICING’s design. This paper’s focus is ICING’s data plane, specifi- cally interrealm transit. §7 gives example uses of ICING.
2.1 Overview
ICING divides the network into realms. Realms are de- fined by trust boundaries, and no two realms need trust each other. ICING does not change the basic network topology and peering model: today’s ASes map naturally to realms. However, the granularity of a realm is variable. An organization that centrally administers its forwarders and hosts may form a single realm. When different ma- chines belong to different people, each host, or virtual machine, or even process can be its own realm. (In §3.4, we describe a more convenient way to name endpoints.) For deploying ICING, it may be useful to regard the cur- rent Internet as one realm. A packet’s route brings it from the source, through intermediate realms, to the destination. We call this se- quence of realms a path. The essence of ICING is as follows: Each realm decides whether a path, or set of paths, conforms to its security policy. These decisions
mation that is needed, rather than heuristics. These three requirements imply that all entities in a packet’s path should make an allow/deny decision in the control plane, prior to packets flowing, based on arbitrary factors. But why is ICING’s data plane de- signed to enforce consent-to-connectivity? While any en- tity’s policies are likely to be idiosyncratic, any policy based in part on the path requires that the forwarding infrastructure constrain packet flow to meet the policy. And, as mentioned in §1, such policies are common; in the allow/deny decisions in prior work, entities of- ten use some component of the path as an input, such as the source [6, 8, 13, 25, 32, 42, 44], the two end- points [18, 24, 36], a suffix [41], a prefix [9], an arbitrary subsequence [22], the whole path [34], etc. We further discuss consent-to-connectivity in §7.
3 Design of ICING’s data plane
We start with a simplified version of the protocol that assumes that each realm consists of a single forwarder. We then augment the protocol with vnodes, a mechanism that enables delegation of PoC-minting ability, bootstrap- ping, intrarealm forwarding, and more.
3.1 Threat model
We assume that some realms (end-hosts and providers) are controlled by attackers who are subject to standard cryptographic hardness assumptions. We refer to the realms they control as malicious. These realms can de- viate arbitrarily from the protocols that we describe, in- cluding sending arbitrary packets or flooding the links they have direct access to. Realms that obey the protocol we term honest. Hon- est realms can carry packets between other realms. We make no assumptions on how malicious realms are im- plemented (they may directly connect to one another and be controlled by a single attacker). ICING concerns the behavior of honest realms, in particular determining when they have carried or should carry a packet.
3.2 Basic protocol
To uphold consent-to-connectivity, the data plane en- sures that packets may transit an honest realm R only under the following conditions:
- [Path Validity] The path P in the packet’s header must have been previously approved by R; and
- [Provenance Verification] The packet must verifiably have transited all honest realms before R in P.
Note that neither condition explicitly constrains the tra- jectory packets take after leaving a given realm R. Taken together, however, they guarantee that packets that skip any honest realm will be dropped at the next honest realm that they traverse. In other words, the two conditions to- gether imply the following:
P 〈R 0 , R 1 , R 2 ,... , Rn− 1 , Rn〉. A packet’s path. M {vers, ctr, proto, len, payload}. Its contents. Ri A public key which is also the realm name. xi The private key of realm Ri. si The symmetric PoC key used by Ri’s forwarders to verify packets. ki,j SHA- 1 (Ri, Rj, gxixj^ ). A symmetric key shared by realms Ri and Rj, and derived from their names using non-interactive Diffie-Hellman key exchange. pocP,i CMAC(si, P). Proof of consent (PoC) to path P by realm Ri. Vi^ 〈Vai 1 , Vb 1 , Vai 2 , Vb 2 ,... , Vain, Vbn〉. Auth vector when packet leaves Ri; allows downstream realms to verify provenance. Aj H′^
pocP,j, H([P, ] M)
Va^0 j PRF- 96 (k0,j, {[H(P, M), ] first 96 bits of Aj}). Vaij PRF- 96 (ki,j, {[H(P, M), ] Vai j− 1 }). Proves to Rj that packet has transited path through Ri. Unused if i ≥ j. Vbj Last four bytes of Aj. Guards the forwarder slow path from DoS attacks.
Figure 2—Cryptographic values in ICING. PRF-96 is AES- XCBC-MAC-96 [21]. H′^ is CMAC (but may change in the fu- ture). Hash H and the bracketed quantities in A and Va were not implemented for the benchmarks in §6.
- [No Path Deviation] Packets that fork off their de- clared path P by skipping an honest realm Rskip cannot traverse any honest realm that succeeds Rskip in P. Designing a viable protocol that enforces the above conditions requires some care: in particular, per-packet public-key operations would induce an unacceptable per- formance penalty. Our protocol avoids this penalty by sharing symmetric keys between each pair of realms. Es- tablishing the shared keys without adding hard state to forwarders requires public-key cryptography but only the first time the two given realms appear on the same path (or after a symmetric key has been evicted from cache).
Preliminaries and notation. As mentioned above, the name of a realm, Ri, is a public key. Because public keys are carried in packets, we wanted them to be as small as possible. Thus, we chose to use elliptic curve cryptogra- phy. Every Ri is a point on NIST’s B-163 [5], which is a binary-field elliptic curve group. The corresponding pri- vate key, xi, is the discrete logarithm of the public key: gxi^ = Ri, where g is a globally agreed upon generator. 2 To make our protocol more amenable to a hardware
(^2) The elliptic curve literature uses additive notation, but here, for readability, we use the more familiar multiplicative notation.
function SENDPACKET(P, pocs, m) // P = 〈R 0 , R 1 ,... , Rn〉 // pocs =
˘ pocP,i = CMAC(si, P) | 1 ≤ i ≤ n
¯ // m = {proto, packet-len, [return path + PoCs, ]data} // M = vers||ctr||m for (i = 1... n) do Ai = H′(pocP,i, H([P, ] M)) Vai = PRF-96(k0,i, {[H(P, M), ] first 96 bits of Ai}) Vbi = last 4 bytes of Ai V^0 = 〈Va 1 , Vb 1 , Va 2 , Vb 2 ,... , Van, Vbn〉 path-idx = 1 pkt = ver||path-len||path-idx||cntr||P||V^0 ||m transmit pkt to R 1 // may need intrarealm forwarding cntr++
Figure 3—Pseudocode for packet construction: S = R 0 con- structs a packet to send payload m along path P. Note that P is 0-indexed and V^0 is 1-indexed.
implementation, we reduce the representation of the key from 163 bits to 160 bits. We do so by requiring the top 3 bits of Ri to equal a hash of the rest of Ri; the cost is a factor of 8 in expected key generation time. The secu- rity attained is roughly 80-bit security, comparable to the security of RSA-based systems with 1024-bit moduli [5]. We label the source of a packet R 0 , the destination Rn, and the path P = 〈R 0 , R 1 , ..., Rn〉. For each Ri, as- sume the source holds a PoC that proves to forwarders in Ri that Ri has consented to P. For the purpose of creating PoCs, every realm Ri has a symmetric key, si, known to the equipment in that realm. A PoC is set as pocP,i = CMAC(si, P). (We use CMAC-AES-128 [4], a deterministic message authentication code with pseudo- random outputs.) Realms change their PoC keys, si, pe- riodically to safeguard against chosen-message cryptan- alytic attacks. Each realm implicitly has a pairwise shared key with every other realm. The keys are generated by non- interactive Diffie-Hellman key exchanges: realms Ri and Rj share ki,j = SHA- 1 (Ri, Rj, gxixj^ ).^3 The purpose of these keys is for realms along a packet’s path to provide each other with cryptographic evidence of packet provenance. Figure 2 summarizes the protocol’s constructs.
Packet construction by sources. To send a packet, a source follows the pseudocode in Figure 3. Sources do not place PoCs in packets directly (see Figure 1): the presence of a PoC, say pocP,i, in the packet would only inform realm Ri’s forwarders that the realm consented to the purported path, P, listed in the packet. However, there
(^3) Realms also use their private keys to sign certificates (§4). Such “dual purposing” of key material is wisely discouraged by folklore be- cause the resulting interplay among distinct cryptographic functions might void their individual guarantees. A careful security analysis in- dicates that ICING’s protocol is safe in this regard, but the details are outside this paper’s scope.
function RECEIVE(pkt) // pkt = vers||path-len||path-idx||cntr||P||Vi−^1 ||m // M = vers||ctr||m pocP,i = CMAC(si, P) Ai = H′(pocP,i, H([P, ] M)) // extract components in Vi−^1 that we need to verify let 〈Vai i− 1 , Vbi〉 = the ith entry in Vi−^1 // following line protects slow path from spurious calls check that Vbi equals last 4 bytes of Ai: if not, drop // following line may require slow path invocation compute k0,i, k1,i,... , ki−1,i // simulate what earlier forwarders should have done to // the ith component of the authorization vector W = first 96 bits of Ai for 0 ≤ j ≤ i − 1 do W = PRF-96 (kj,i, {[H(P, M), ] W}) check that W = Vai i− 1 : if not, drop // following line may require slow path invocation compute ki,i+ 1 ,... , ki,n // construct Vi Vi^ = Vi−^1 for i + 1 ≤ j ≤ n do Vaij = PRF-
` ki,j,
˘ [H(P, M), ] Vai j−^1
¯´
increment pkt.path-idx to i + 1 transmit pkt to Ri+ 1 // may need intrarealm forwarding
Figure 4—Pseudocode for packet forwarding: Ri validates pkt and transforms Vi−^1 to Vi^ before sending pkt to the next realm. Note that P is 0-indexed and Vi−^1 , Vi^ are 1-indexed.
would be no guarantee that the packet actually took this path, and nothing to constrain the packet to continue to follow the path. To provide such a guarantee, the packet contains an authorization vector, Vi. Each realm Ri receives a packet with Vi−^1 and transforms the vector to Vi. The source constructs V^0 from a set of packet- and realm-specific authenticators, A 1 ,... , An. The authenticators bind the PoC, path, and payload together in a concise value that can be certified by each realm on the packet’s path. Intu- itively, V^0 j assures realm Rj that realm R 0 claims to have sent this packet because only realms R 0 and Rj know the key k0,j used to certify this value.
Packet verification and transformation by for- warders. Figure 4 depicts the handling of packets by a realm, here represented as a single forwarder. Con- sider the ith entry of Vi−^1 , the vector at the time that Ri receives the packet. It is in two pieces: Vai i− 1 and Vbi. The first piece, Vai i− 1 , should, in the absence of attack, be a recursive application of PRF-96 to Ai under keys k0,i,... , ki−1,i. Rj can verify Vai i− 1 by recomputing it; if the value is correct, Ri knows all previous realms in the packet claim to have carried the packet, so at least the subset of them that is honest will have carried the packet.
P Path is now 〈T 0 , T 1 , T 2 ,... , Tn− 1 , Tn〉. Ti A pair (Ri, ri). Ri is a realm; ri is a vnode within Ri. sTi The PoC key for vnode Ti = (Ri, ri). POCP,i CMAC(sTi , P).
Figure 5—Modification to cryptographic constructs in §3; compare to Figure 2.
Each vnode has its own PoC key, which simplifies changing PoC keys—one can gradually migrate traffic to a new vnode before changing the old one’s key. A realm may chose to divulge the PoC key for one of its vnodes to another realm, thereby delegating the ability to create PoCs. We expect providers to give PoC keys to customers who run their own path servers. Customers may then deny upstream PoCs according to their own policy, stopping unwanted traffic before it transits their provider’s networks. A backbone provider will likely give PoC keys to the ISPs it sells transit to. Those ISPs may wish to subdele- gate the backbone’s PoC key. To allow controlled delega- tion, a backbone provider may delegate a block of vnodes to each ISP, who can then further subdelegate them in- dividually or in smaller blocks. Like Platypus [36], we generate all vnode keys for a given prefix from a sin- gle master key in a way that allows further subdelegation along bit boundaries. (If sp is the master key for prefix p, then sp||b = H(sp, b).)
3.5 Bootstrapping
One particularly important application of delegation is bootstrapping. Realms will want to host their own control-plane servers, in which case potential senders need a PoC to contact the server that grants PoCs. For this purpose, each realm can create a bootstrap vnode that connects only to its PoC server. Bootstrap vnode keys can be published in naming service glue records (much like DNS). Anyone possessing such a key can create PoCs to reach the realm’s PoC server. Since the bootstrap vnode does not connect to any other networks, knowledge of its PoC key does not allow non-bootstrap traffic to transit the realm. We discuss potential “Denial of PoC” attacks [10] in §4.1.
4 Control plane functions
Our focus in this paper is ICING’s data plane. For con- text and concreteness, we now mention some control plane functions. We have space to give only a sketch. The sketch covers only the “common case”; many other uses are possible. Configuration. When an ICING host joins a network, it receives from a local configuration server the fol- lowing state: (1) partial interrealm paths to and from
upstream realms. For example, the configuration server might give a host at a university partial paths to a local provider that peers with the university, to the Internet network, to the university’s commercial ISP, and to the top-tier ISP from which the commercial ISP buys ser- vice (as well as the reverse directions of those partial paths). (2) For each realm in the partial path, a PoC key that allows the host to mint PoCs for those realms. The idea here is that each provider would have sub-delegated (§3.4) a slice of vnode space to each of its customers. At the extreme, an end-host has permission to mint PoCs that allow the end-host’s traffic to flow over, for exam- ple, a “slice” of a backbone network. (3) The full path, and accompanying bootstrap PoC keys, to at least one path server, PS 0. Path construction. When an ICING host wants to reach a desired destination (steps 2–4 at the end of §2.1), it submits a human-friendly name to PS 0. The host and PS 0 perform a negotiation. If it is successful, they will have identified a path that goes through an intermediate realm I such that: (a) the host can mint PoCs to reach I and (b) the path server can mint PoCs to get from either I to the destination or from I to the next path server that the ICING host should query, at which point the process continues. (Here, the path server has the ability to mint PoCs on behalf of intermediate realms just as the sender does.) This approach is inspired by NIRA [43]. Consent servers and consent certificates. So far, we have described senders and path servers as minting PoCs. But in fact, each realm has a consent server (CS) that takes as input proposed paths and ancillary information, and issues PoCs on behalf of the realm. The ancillary in- formation is arbitrary. The option that we envision (and have implemented) is for the CS to require a set of con- sent certificates from each realm along the proposed path. Such certificates are signed by a realm’s private key. An honest CS never issues a PoC unless it gets a set of consent certificates from each realm along the proposed path. Thus, packets with non-conforming paths will not travel further than the first honest realm along the path. Topology propagation. The topology propagation protocol that we have designed “pushes information to the edges” (i.e., to path servers and configuration servers). We do not have space for more than a few words. Under this protocol, advertisements flow down- ward, from transit providers to customers. A provider, say T, collects the advertisements that it has heard, ap- pends a group of certificates that express information about T’s interrealm links, and forwards the augmented set of advertisements to its transit customers. Thus, when a realm receives an advertisement, it gets information about its provider, its provider’s peers, its providers’ providers, etc. This process can handle topology changes at coarse grain time scales. There are several possibili-
ties for handling dynamic topology changes at fine grain time scales, but we have not implemented them, and there are some issues that require care (as also observed in [11, 12]) so leave this to future work for now; see §9.
4.1 Flooding attacks
If a realm wishes to receive traffic from anyone in the world (for instance because it runs a Web server), that realm will be vulnerable to packet-flooding DoS attacks. In the general case, such attacks are inevitable and im- possible to defend against. ICING cannot solve the gen- eral problem, but the specifics of a scenario often leave room for effective defense measures; ICING allows vic- tims to capitalize on those specifics. We give three scenarios in which ICING can help. The DoS victim may have user accounts, only a small frac- tion of which are compromised (even if many machines attack). The attacks may be coming through a small num- ber of providers. Or the attacker’s machines may mostly reside behind honest forwarders and path servers. However, we must consider two types of DoS attack— a flooding attack against an application server, and a “denial of PoC” attack against the victim’s consent server, which may be located on a bootstrap vnode with a publicly known PoC key. DoS attacks on application servers can be mitigated by withholding consent (not granting PoCs), a common approach [8, 32, 42, 44]. ICING contributes the ability to base decisions on the full path, allowing victims to recognize and deny a malicious provider emulating many customers. If the victim has some notion of authenticated users, each user can be redirected to a different vnode and fair queuing used among vnodes to ensure a small number of bad users cannot crowd out the honest ones. Several measures can combat “denial of PoC” attacks in which attackers flood a realm’s consent server with traffic. We first note that a consent server can be reach- able though multiple vnodes. Some will likely be public, while others may be reserved for employees (who might cache PoC keys on their laptops), and so with fair queu- ing at least an attack on the public vnode will not crowd employees out from reaching the company. Alternatively, an organization need not host its own public consent server. Today, third-party DoS defend- ers exist that mediate all traffic to customers [35]. With ICING, such services need only host (or mediate access to) the public consent server. Authenticated users can then communicate directly with the organization.
5 Implementation
We describe our implementation of the data plane and then our (partial) control plane implementation. Data plane. Our prototype ICING forwarder accepts ICING packets carried in Ethernet frames. The core of
Figure 6—Hardware logic area costs.
the prototype is an image that runs on the NetFPGA pro- grammable hardware platform [3], which has 4 Gigabit Ethernet ports. The hardware executes the fast path (see Figure 4 in §3.2). When an ICING packet enters the fast path, if the packet’s path contains one or more realms Rj for which the forwarder, representing realm Ri, does not have ki,j (§3.2) cached in hardware, the hardware sends the packet to a software slow path (again, see Figure 4) over the PCI bus to an x86 processor running Linux (ver- sions 2.6.18 and 2.6.25 in our experiments). The slow path, implemented in the Click modular soft- ware router, calculates the needed keys and installs them in the hardware’s key cache, possibly evicting old keys. The Diffie-Hellman is implemented with the publicly available MIRACL cryptographic library. Together, the fast path and software implement the protocol described in §3.2. We have not yet implemented per-vnode PoC keys (§3.4) and plan to do so soon. The hardware image is based on the reference base package provided by the NetFPGA project. We imple- mented the ICING-specific logic, including the crypto- graphic modules and re-used the reference package’s support modules—Ethernet logic, DMA logic, queueing, and the interface to host software. The total equivalent gate count for our 2 Gbit/s NetFPGA forwarder design is 11.1M gates. In compar- ison, the 4 Gbit/s reference IP router from the NetFPGA project [3] has an equivalent gate count of 8.65M. The implementation uses 78% of the total FPGA logic area. The relative costs in terms of logic area, or lookup ta- bles (LUTs) used, of the main components are shown in Figure 6. Of the 78% total, the ICING-specific logic uses approximately 46.7%, while the support modules men- tioned earlier use the other 53.3%. Figure 7 shows the major hardware blocks. We now address an important performance concern. From Figure 4, one might expect the cost of processing a packet to depend on the length of a packet’s path and on a realm’s position in the path (the value of i): a larger path length means more iterations of AES-XCBC-MAC- 96, and the value of i affects whether the forwarder is doing more “verification” or more “MACing”. However, the number of AES-ENCRYPT operations—the slowest
Payload Size Path Len Path Idx 0
1
2
3
Parameter changed
Throughput (Gbit/s)
Min Avg Max
Figure 8—Maximum, average, and minimum throughputs when each parameter was varied as described in §6.1. For each 30-second sample, the standard deviation was less than 5% of the mean. The throughput varies slightly as packet parameters are changed.
packets, while for latency measurements the payload size was set to zero. When the path index was a parameter, it varied over {1, 5, 10,15, 18} and the path length was set to 20 and the packet size to 831 bytes. When the packet size was a parameter, it varied over {311, 567, 823, 1335, 1514 } while keeping the path length and path index con- stant at 7 and 3 respectively.
6.2 Packet overhead
The ICING header size is significant. The header fields that are not dependent on the packet’s path length use 13 bytes (see Figure 1). Each (Ri, ri) is 24 bytes, and each component of V uses 16 bytes (the notation is from Fig- ures 2 and 5). Thus, the total overhead in bytes is:
13 + 24 + 40 · (path length − 1 )
For a packet whose path is 7 realms long—the average length of an AS level path found in [28]—the header is 277 bytes or 18.3% of a 1514-byte packet. By compari- son, IP’s 20-byte header is 1.3% of a 1514-byte packet.
6.3 Microbenchmarks
ICING forwarder We began with our prototype’s throughput and per-packet latency, for both the fast path and the slow path. As mentioned in §5, we expect the first two metrics to be independent of the path length and of the forwarder’s position, i or the path index, in the path (except as far as these factors affect the total packet length). Forwarder fast path throughput. In our lab, we connected the four ports of the ICING forwarder to a NetFPGA packet generator that can send ICING packets at line-rate. We varied the packet’s path length, path in- dex, and the payload size and measured the throughput over 20 30-second samples at the measurement points described in §6.1. The ICING forwarder looped ingress packets back into the packet generator, which measured the average rate of packet arrivals. All required symmet- ric keys were already in the hardware cache.
0 500 1000 1500 0
10
20
30
Packet size
Latency (
μs)
Path len varied Payload varied Path idx varied Trend line
Figure 9—Summary of the three latency measurement results using the points described in §6.1. The standard deviation was 2% less than the mean in all measurements. The prototype’s forwarding latency is independent of the ratio of path size to payload size or current path index. The latency depends only on total packet size; the trend line has slope 15 ns/byte.
Figure 8 plots the measurement results.The standard deviation of each of the 20 samples we collected was less than 5% of the mean. The minimum aggregate throughput—over the four ports and including the Eth- ernet preamble and inter-packet gap—was 2.1 Gbit/s. Forwarder fast path latency. To measure latency, again we employed the packet generator and used the points described in §6.1. The packet generator can accu- rately send packets simultaneously and timestamp packet arrivals (with errors of a few nanoseconds). In our lab, we connected two ports of the packet generator together through the ICING forwarder and the other two ports di- rectly to each other. All required symmetric keys were again in the hardware cache. The packet generator simultaneously sent packets through the ICING forwarder and through the loop-back cable and timestamped packet arrivals. The difference between the timestamps is the forwarder latency. We sampled 100 single packet latencies for each mea- surement point. Figure 9 shows that latency grew linearly with packet size and was independent of the forwarder’s position in the packet path, as hypothesized in §5. Forwarder slow path latency. We do not consider the time a packet takes to travel from the fast path to the slow path via the PCI bus or vice versa because the PCI bus latency is negligible (tens of microseconds) compared to the shared key calculation as will be shown. The primary function of an ICING forwarder’s slow path is the calculation of the shared symmetric keys (ki,j) that are not currently in the hardware cache. To measure the cost of this operation, we ran the shared key calcula- tion function in a tight loop to calculate 3, 000 keys. We ran the calculations on the slow machine (§6.1) because both end-hosts and forwarders will need to get shared keys in an ICING network. On average, a single key cal- culation took 4 ms, with a standard deviation of 43 μs. Writing this new key to hardware cache was negligible in comparison, taking approximately 9 μs. Because elliptic curve cryptography (ECC) is
0 10 20 30 40 0
10
20
30
Path length
Latency (
μs)
Avg Trend
Figure 10—Average latency of the example consent server as the requested paths’ lengths is varied. Standard deviation was less than 2% of the mean. The trend line has slope 0.648 μs/realm
amenable to a hardware implementation, it is possible to considerably speed up the slow path using a hardware ECC co-processor.
Path server and consent server To better represent the hardware that would run ICING control plane servers, we used the fast Internet2 nodes (§6.1) to run server bench- marks. We measured the throughput of our prototype path server and consent server by isolating the two bot- tleneck functions: getpath and getpocs. Our prototype getpath function generates complete paths by finding a common intermediate realm (as de- scribed in §4). We bootstrapped the path server with 20 paths from intermediate realms to destination realms, and called its getpath function providing it a random num- ber between 1 and 20 of paths from sources to interme- diate realms. We measured the throughput of getpath by calling it in a tight loop over 20 30-second inter- vals. The prototype path server generated approximately 6,990 paths/s, with a standard deviation of 42.6 paths/s. We measured the throughput of the getpocs function of a consent server by calling it in a tight loop and vary- ing the path length in each request. Since the consent server is single-threaded, we plot the latency of a sin- gle request as the inverse of the throughput. The plot in Figure 10 shows that the latency to generate a PoC is pro- portional to the path length, which is expected because the PoC is a CMAC of the path.
End-host We measured the throughput at which senders can create and receivers can verify packets. For these ex- periments, we used the medium1 machine (S6.1). Sender. We measured the sender latency as the path length varied when it already had all the required PoCs. The sender generated 1000 1514-byte sized packets in a tight loop with variable path length as in §6.1. Receiver. We also measured receiver latency as the path length varied using the same parameters. We sent the same packet to the receiver (in the same memory lo- cation) in a tight loop and measured the latency for 1000 packets. The results of both the sender and receiver la- tency measurements are in Figure 11.
0 10 20 30 40
0
200
400
600
800
Path length
Latency (
μs)
Sender^ Receiver
Figure 11—Average latency of senders and receivers when the total packet size was constant at 1514 bytes and the path length changes using the path length measurement points from §6.1. Standard deviation was less than 2% of the mean at each point. The sender has the worst latency when the number of encryp- tions is maximized at path length = 20. The receiver latency is proportional to the path length with a slope of 0.55 μs/realm.
Figure 12—Our Internet2 experimental deployment. Two end- hosts in geographically distant locations are connected over the Internet to a path server and to Internet2 nodes running as ICING forwarders. The receiver is similar to a forwarder and incurs la- tency proportional to the path length for a constant packet size. The sender, on the other hand, CMACs the payload path length times. And since the payload size is the dif- ference of the packet size and path size, it follows that, as the path length increases, the latency follows a parabola as shown in Figure 11.
6.4 End-to-End
In our end-to-end experiment, each node is a unique realm in an attempt to emulate a microcosm of three peering network providers sharing a single path and con- sent server. Figure 12 shows our deployment. We ran forwarders on the fast Internet2 machines and used the medium1 and medium2 machines as sender and receiver respectively (see §6.1). The Los Angeles Internet2 node ran a path server that could be queried by ICING-enabled end-hosts over IP. The end-hosts connected to the back- bone using a software IP tunnel. The first packet of a new flow experiences two ICING delays: path/PoC query and slow path invocation. To measure these two operations, we use an ICING ping ap- plication on the end-hosts. The sender initiates the ping by first querying the path server for a path to the end- host at the other end of the Internet2 network. The path server acts as both a path and consent server, returning a complete path and the necessary PoCs for communi-
7.2 Non-security applications
Under ICING, the existence of multiple paths from clients to servers is not hidden. One consequence is that ICING provides a natural solution to “multipath”, i.e., permit- ting sources to choose from multiple paths to a destina- tion or to use several paths simultaneously. Another consequence is that ICING facilitates anycast. Path multiplicity is already explicit in ICING, and the mechanism works whether the paths terminate at the same or different physical instances. Thus, clients and servers can negotiate between them to devise optimal routes to nearby replicas. Today, commercial content dis- tribution networks approximate this functionality with DNS tricks, but doing so requires detailed, proprietary knowledge of the Internet’s topology.
7.3 Potential unifications
We now try to give some intuition for how ICING can express the high-level goals—except for backward compatibility—of current mechanisms and prior work. Consider blacklisting. One can implement it unilater- ally in ICING’s control plane servers by withholding con- sent and not returning PoCs for any path that contains a blacklisted realm. More interesting, using delegation via vnodes [22] (see §3.4), a destination realm can get other realms to drop blacklisted packets closer to the source. As discussed in §3.4, if providers delegate PoC creation for each vnode to the path server of the customer billed for that vnode, then a destination’s path server can with- hold upstream consent for blacklisted paths. Consider VPNs, in which an organization wants only employees to send packets to its network. Under ICING, the organization simply sets its consent and path servers not to return paths or PoCs to non-employees. Doing so requires a way for employees to authenticate themselves to the company’s consent servers (e.g., single-sign-on au- thentication systems). But ICING can also permit richer and more precise policies than VPNs. For instance, each employee may be given a separate vnode which can reach only an employee-specific set of servers. Note that in this example, the organization’s consent server is likely to be a target of denial-of-service attacks, but it can be located off-site, possibly hosted by a DoS- prevention company [35]. Or an organization might dis- pense with consent servers and instead issue employ- ees tamper-resistant hardware devices containing present and future PoC keys. So long as each employee is given a separate vnode, revocation is simple. A number of recent proposals have shown how to derive security benefits from scenarios in which edge routers are honest even if the hosts behind them are com- promised [8, 13, 25, 32, 42, 44]. ICING offers similar benefits when a host’s local path resolver (§4) is hon- est, as follows. If a host is “undesirable” in the view of
some destination, then its local resolver will not be able to negotiate a valid path from it to the destination. With no valid path, an honest resolver will refuse to generate PoCs that allow the host to send traffic out of the local network along that path; this refusal “contains” the “un- desirable” host, in the view of the given destination. Crude taxonomy. We now give a taxonomy of pro- posals (which is necessarily incomplete) and show how ICING captures this taxonomy. By upholding consent-to- connectivity, ICING captures the cross product of which entity along the path gets to make a decision with which entities along the path form the basis of the decision. That is, consent-to-connectivity captures senders con- trolling the path [34], receivers deciding which hosts can send them traffic [8, 13, 25, 32, 42, 44], providers controlling inbound paths [9], providers controlling on- ward paths [41], providers controlling subsequences of paths [22], middleboxes controlling each other [24], etc. And, this cross-product directly suggests new security notions that are not expressible or enforceable under the status quo. For example, receivers should be able to con- trol the path toward them. This would allow an organiza- tion (financial, medical, intelligence, legal) that handles sensitive data to ensure that only providers that it trusts carry data to it or between two remote offices.^5 Of course, many allow/deny decisions today are based on packet contents rather than some component of the path (e.g., “packet washing”, intrusion detection, or mal- ware detection). ICING permits such defenses. Within an organization, it suffices to configure the vnode topology to force traffic through these middleboxes. More interest- ingly, however, such defenses need not be implemented inside the realm that wants them. A company might con- sent only to inbound traffic whose path transits some third-party security service. The service may be many hops away from the company on the network, but if the service appears in the path, then any honest realm be- tween the service and the company will drop forged traf- fic that has not transited the service. Note that we do not discuss “control plane” security proposals (e.g., [26, 29, 40]); the issues addressed by these proposals don’t arise under ICING. Also, given our focus on the network layer, we do not discuss proposals that work at other layers (e.g., DNS, overlays [7, 17, 30]).
8 Other related work In §7.3, we referenced many related works during our argument that ICING unifies the high-level goals of many prior proposals. Here we just single out a few research strands that ICING borrows from or builds on. ICING’s PoCs generalize network capabilities [8, 36, (^5) Such an organization might not want to rely on end-to-end encryp- tion alone because, even with encryption, network eavesdroppers can sometimes learn useful information from packet sizes and timing [38].
42, 44] and Visas [18]. In all cases, some logical entity “mints” a permission that a sender can use for packet transit. At a very high level, the principal differences be- tween a PoC and a capability or a Visa are that (1) PoCs explicitly consent to the entire path that a packet is sup- posed to take and (2) the forwarding infrastructure con- strains the packet to take this path. These requirements permit ICING to uphold consent-to-connectivity but lead to a very different design that, compared to these propos- als, is a harder “deployment sell”. ICING’s approach to delegation is inspired by Path- let routing [22] (from which we borrow vnodes) and Platypus [36] (whose hierarchical subdelegation tech- nique ICING uses for delegating “slices” of vnode space). ICING also borrows some of its control plane techniques from NIRA [43]. And ICING’s instantiation of policy in general-purpose servers apart from forwarding hardware echoes [14, 15, 23]. As mentioned in §7.1, a natural consequence of ICING is that packets’ paths respect providers’ policies. Of course, there has been much work in policy routing over the years from early frameworks like [16, 19, 20] through more recent attention to BGP (e.g., [41]), but none of these proposals upholds consent-to-connectivity. ICING is trying to solve a variant of the “secure for- warding” problem [11, 12, 33, 34]. In contrast to this work, ICING does not rely on a trusted entity or central- ized administration (as in [33, 34]). ICING’s recursive ap- plication of PRF-96 (§3.2) is reminiscent of a construc- tion in [12], but ICING operates under a different trust model and cannot assume pairwise coordination. Also, in contrast to these proposals, ICING explicitly upholds all stakeholders’ interests, not just the source’s. Of course, many “clean slate” proposals have emerged recently. ICING’s relationship to this work is that ICING respects traditional layering (mostly) and is a network layer so is mostly orthogonal to proposals that focus on other layers (e.g., [31, 39], etc.).
9 Summary and future work
Summary. Our purpose in this work was exploratory: to identify a coherent security property and to find a way to uphold it with a viable implementation; we believe this effort was successful. Near-term work. In the data plane, we will enhance our prototype to implement the bracketed quantities in Figure 2 and per-vnode keys (§3.4). In the control plane, we will run control traffic over ICING, improve our path server, and handle route failures and topology changes at fine grain. Last, we will consider how to use the momen- tum behind various clean slate efforts to identify poten- tial ICING deployments.
References [1] The 32-bit autonomous system number report. http://www.potaroo.net/tools/asn32/index.html. Last accessed on 3/2/2009. [2] Integrated device technology (IDT) quick reference guide. http: //www.idt.com/products/getDoc.cfm?docID=18640144. Last accessed on 1/30/2009. [3] NetFPGA: Programmable networking hardware. http://netfpga.org. [4] Recommendation for block cipher modes of operation: The CMAC mode for authentication. Special Publication (800-series), May 2005. DRAFT FIPS PUB 186-3. [5] Digital signature standard (DSS). Federal Information Processing Standards Publication, November 2008. DRAFT FIPS PUB 186-3. [6] D. Andersen, H. Balakrishnan, N. Feamster, T. Koponen, D. Moon, and S. Shenker. Accountable Internet protocol. In SIGCOMM, Aug. 2008. [7] D. G. Andersen. Distributed filtering for Internet services. In Proc. USENIX Symposium on Internet Technologies and Systems (USITS), Mar. 2003. [8] T. Anderson, T. Roscoe, and D. Wetherall. Preventing Internet denial-of-service with capabilities. In HotNets, Nov. 2003. [9] K. Argyraki and D. R. Cheriton. Loose source routing as a mechanism for traffic policies. In Proc. SIGCOMM Workshop on Future Directions in Network Architecture, Sept. 2004. [10] K. Argyraki and D. R. Cheriton. Network capabilities: The good, the bad and the ugly. In HotNets, Nov. 2005. [11] I. Avramopoulos, H. Kobayashi, R. Wang, and A. Krishnamurthy. Amendment to: Highly secure and efficient routing. Feb. 2004. [12] I. Avramopoulos, H. Kobayashi, R. Wang, and A. Krishnamurthy. Highly secure and efficient routing. In INFOCOM, Mar. 2004. [13] H. Ballani, Y. Chawathe, S. Ratnasamy, T. Roscoe, and S. Shenker. Off by default! In HotNets, Nov. 2005. [14] M. Caesar, D. Caldwell, N. Feamster, J. Rexford, A. Shaikh, and J. van der Merwe. Design and implementation of a routing control platform. In NSDI, May 2005. [15] M. Casado, M. Freedman, J. Pettit, J. Luo, N. McKeown, and S. Shenker. Ethane: Taking control of the enterprise. In SIGCOMM, Aug. 2007. [16] D. Clark. Policy routing in internet protocols. RFC 1102, May
[17] C. Dixon, T. Anderson, and A. Krishnamurthy. Phalanx: Withstanding multimillion-node botnets. In NSDI, Apr. 2008. [18] D. Estrin, J. Mogul, and G. Tsudik. VISA protocols for controlling inter-organizational datagram flow. IEEE JSAC, 7(4), May 1989. [19] D. Estrin, Y. Rekhter, and S. Hotz. Scalable inter-domain routing architecture. In SIGCOMM, Aug. 1992. [20] D. Estrin and G. Tsudik. Security issues in policy routing. In Proc. IEEE Symposium on Security and Privacy, May 1989. [21] S. Frankel and H. Herbert. The AES-XCBC-MAC-96 algorithm and its use with IPsec. RFC 3566, Network Working Group, September 2003. [22] P. B. Godfrey, S. Shenker, and I. Stoica. Pathlet routing. In HotNets, Oct. 2008. [23] A. Greenberg, G. Hjalmtysson, D. A. Maltz, A. Myers, J. Rexford, G. Xie, H. Yan, J. Zhan, and H. Zhang. A clean slate 4D approach to network control and management. ACM CCR, 35(5), Oct. 2005. [24] S. Guha and P. Francis. An end-middle-end approach to connection establishment. In SIGCOMM, Aug. 2007. [25] M. Handley and A. Greenhalgh. Steps towards a DoS-resistant Internet architecture. In Proc. SIGCOMM Workshop on Future Directions in Network Architecture, Aug. 2004.
