Network Coordinate System

From HandWiki
Revision as of 17:24, 6 February 2024 by Len Stevenson (talk | contribs) (change)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

A Network Coordinate System (NC system) is a system for predicting characteristics such as the latency or bandwidth of connections between nodes in a network by assigning coordinates to nodes. More formally, It assigns a coordinate embedding [math]\displaystyle{ \vec c_n }[/math] to each node [math]\displaystyle{ n }[/math] in a network using an optimization algorithm such that a predefined operation [math]\displaystyle{ \vec c_a \otimes \vec c_b \rightarrow d_{ab} }[/math] estimates some directional characteristic [math]\displaystyle{ d_{ab} }[/math] of the connection between node [math]\displaystyle{ a }[/math] and [math]\displaystyle{ b }[/math].[1]

Uses

In general, Network Coordinate Systems can be used for peer discovery, optimal-server selection, and characteristic-aware routing.

Latency Optimization

When optimizing for latency as a connection characteristic i.e. for low-latency connections, NC systems can potentially help improve the quality of experience for many different applications such as:

  • Online Games
    • Forming game groups such that all the players are close to each other and thus have a smoother overall experience.[2]
    • Choosing servers as close to as many players in a given multiplayer game as possible.[3]
    • Automatically routing game packets through different servers so as to minimize the total latency between players who are actively interacting with each other in the game map.[citation needed]
  • Content delivery networks
    • Directing a user to the closest server that can handle a request to minimize latency.[2][4]
  • Voice over IP
    • Automatically switch relay servers based on who is talking in a few-to-many or many-to-many voice chat to minimize latency between active participants.[5]
  • Peer-to-peer networks
    • Can use the latency-predicting properties of NC systems to do a wide variety of routing optimizations in peer-to-peer networks.
  • Onion routing networks
    • Choose relays such as to minimize the total round trip delay to allow for a more flexible tradeoff between performance and anonymity.[5][4]
  • Physical positioning
    • Latency correlates with the physical distances between computers in the real world. Thus, NC systems that model latency may be able to aid in locating the approximate physical area a computer resides in.[citation needed]

Bandwidth Optimization

NC systems can also optimize for bandwidth (although not all designs can accomplish this well). Optimizing for high-bandwidth connections can improve the performance of large data transfers.[6][7][4]

Sybil Attack Detection

Sybil attacks are of much concern when designing peer-to-peer protocols. NC systems, with their ability to assign a location to the source of traffic can aid in building systems that are Sybil-resistant.[8][9]

Design Space

Landmark-Based vs Decentralized

Almost any NC system variant can be implemented in either a landmark-based or fully decentralized configuration. Landmark-based systems are generally secure so long as none of the landmarks are compromised, but they aren't very scalable. Fully decentralized configurations are generally less secure, but they can scale indefinitely.

Euclidean Embedding

  • This design assigns a point in [math]\displaystyle{ k }[/math]-dimensional euclidean space to each node in the network and estimates characteristics via the euclidean distance function [math]\displaystyle{ d_{ab} = ||\vec c_a - \vec c_b|| }[/math] where [math]\displaystyle{ \vec c_n }[/math] represents the coordinate of node [math]\displaystyle{ n }[/math].
  • Euclidean Embedding designs are generally easy to optimize.
    • The optimization problem for the network as a whole is equivalent to finding the lowest energy state of a spring-mass system where the coordinates of the masses correspond to the coordinates of nodes in the network and the springs between the masses represent measured latencies between nodes.
    • To make this optimization problem function work in a decentralized protocol, each node exchanges its own coordinates with those of a fixed set of peers and measures the latencies to those peers, simulating a miniature spring-mass system where all the masses representing the coordinates of the peers and each mass is connected via a single spring to the node's own "mass" which when simulated, gives a more optimal value for the node's coordinate. All these individual updates allow the network as a whole to form a predictive coordinate space by collaboratively.
  • The laws of Euclidean space require certain characteristics of the distance function to hold true, such as symmetry (measuring from [math]\displaystyle{ a \rightarrow b }[/math] should give the same result as from [math]\displaystyle{ b \rightarrow a }[/math]) and the triangle inequality [math]\displaystyle{ (a \rightarrow b) + (b \rightarrow c) \geq (a \rightarrow c) }[/math]. No real-world network characteristics completely satisfy these laws, but some do more than others[10] and NC systems using euclidean embedding are somewhat accurate when run on datasets containing violations of these laws.[11][12][13]
  • Notable Papers: GNP,[11] PIC[12] Vivaldi,[13] Pharos[14]

Matrix Factorization

  • The matrix factorization design imagines the entire network as represented by an incomplete matrix [math]\displaystyle{ X : \R_{n \times n} }[/math] where [math]\displaystyle{ n }[/math] is the total number of nodes in the network, and any element of the matrix at the intersection between row [math]\displaystyle{ i }[/math] and column [math]\displaystyle{ j }[/math] of the matrix represents a directional latency measurement from node [math]\displaystyle{ n_i }[/math]to node [math]\displaystyle{ n_j }[/math]. The goal is to estimate the numbers in the unfilled squares of the matrix using the squares that are already filled in, i.e. performing matrix completion.[15]
  • To estimate a specific latency between two nodes, this method uses the dot product [math]\displaystyle{ d_{ab} = \vec u_a \vec v_b }[/math] where [math]\displaystyle{ \vec u_n }[/math]/[math]\displaystyle{ \vec v_n }[/math] represents a point in a [math]\displaystyle{ k }[/math]-dimensional inner product space.
  • NC system designs using matrix factorization are generally more complicated than their euclidean counterparts.
    • In the centralized variant, matrix completion can be performed directly on a set of landmarks which have measured latency to every other landmark in a set, thus creating a complete matrix [math]\displaystyle{ X }[/math] representing the landmark network. This matrix can then be factored on a single computer using non-negative matrix factorization (NNMF) into two matrices [math]\displaystyle{ U : R_{n \times r} }[/math] and [math]\displaystyle{ V : R_{r \times n} }[/math] such that [math]\displaystyle{ UV \approxeq X }[/math]. Since matrix multiplication is essentially doing the dot product for each row and column of the input matrices, coordinates for each landmark [math]\displaystyle{ j }[/math] can be represented by two "in" and "out" vectors ([math]\displaystyle{ \vec u_j }[/math] and [math]\displaystyle{ \vec v_j }[/math]) taken respectively from the [math]\displaystyle{ j }[/math]th row of [math]\displaystyle{ U }[/math] and the [math]\displaystyle{ j }[/math]th column of [math]\displaystyle{ V }[/math]. With this, latencies between two landmarks can be approximates by a simple dot product: [math]\displaystyle{ d_{ij} = \vec u_i \vec v_j }[/math]. Any node that wants to figure out their own coordinates can simply measure the latency to some subset of all the landmarks, re-create a complete matrix using the landmark's coordinates, and then perform NNMF to calculate their own coordinate. This coordinate can then be used with any other node (landmark or otherwise) to estimate latency to any other coordinate that was calculated via the same set of landmarks.[15][16]
    • The decentralized variant is decidedly simpler. For a given node, the goal is to minimize the absolute difference (or squared difference) between the measured latencies to the peers and the predicted latencies to the peers. The predicted latency is given by the same equation [math]\displaystyle{ d_{ij} = \vec u_i \vec v_j }[/math] where [math]\displaystyle{ \vec u_i }[/math]is the outgoing vector of node [math]\displaystyle{ i }[/math] and [math]\displaystyle{ \vec v_j }[/math] is the incoming vector of node [math]\displaystyle{ j }[/math]. This goal (or loss function) can then be minimized using stochastic gradient descent with line search.[15]
  • Notable Papers: IDES,[16] Phoenix,[17] DMFSGD[15]

Tensor Factorization

  • Notable Papers: TNDP[18] Leverage Sampling + Personal Devices[19]

Relative Coordinates

  • Notable Papers: RMF[2]

Alternatives

Network Coordinates are not the only way to predict network properties. There are also methods such as iPlane[20] and iPlane Nano[21] which takes a more analytical approach and tries to mechanistically emulate the behavior of internet routers to predict where packets will go and thus what properties a connection will have.

In The Wild

References

  1. ↑ Donnet, Benoit; Gueye, Bamba; Kaafar, Mohamed Ali (2010). "A Survey on Network Coordinates Systems, Design, and Security". IEEE Communications Surveys & Tutorials 12 (4): 488–503. doi:10.1109/SURV.2010.032810.00007. ISSN 1553-877X. https://ieeexplore.ieee.org/document/5462982. 
  2. ↑ 2.0 2.1 2.2 Fu, Yongquan; Xiaoping, Xu (February 2017). "Self-Stabilized Distributed Network Distance Prediction". IEEE/ACM Transactions on Networking 25 (1): 451–464. doi:10.1109/TNET.2016.2581592. ISSN 1558-2566. https://ieeexplore.ieee.org/document/7502073. 
  3. ↑ Agarwal, Sharad; Lorch, Jacob R. (2009-08-16). "Matchmaking for online games and other latency-sensitive P2P systems". Proceedings of the ACM SIGCOMM 2009 conference on Data communication. SIGCOMM '09. New York, NY, USA: Association for Computing Machinery. pp. 315–326. doi:10.1145/1592568.1592605. ISBN 978-1-60558-594-9. https://dl.acm.org/doi/10.1145/1592568.1592605. 
  4. ↑ 4.0 4.1 4.2 Sherr, Micah; Blaze, Matt; Loo, Boon Thau (2009). "Scalable Link-Based Relay Selection for Anonymous Routing". in Goldberg, Ian; Atallah, Mikhail J. (in en). Privacy Enhancing Technologies. Lecture Notes in Computer Science. 5672. Berlin, Heidelberg: Springer. pp. 73–93. doi:10.1007/978-3-642-03168-7_5. ISBN 978-3-642-03168-7. https://link.springer.com/chapter/10.1007/978-3-642-03168-7_5. 
  5. ↑ 5.0 5.1 Sherr, Micah (2009). "Coordinate-based routing for high performance anonymity". Computer and Information Science at UPenn. https://netdb.cis.upenn.edu/papers/msherr-dissertation.pdf. 
  6. ↑ Liao, Yongjun (2013-01-11) (in English). Learning to Predict End-to-End Network Performance. https://orbi.uliege.be/handle/2268/136727. 
  7. ↑ Ramasubramanian, Venugopalan; Malkhi, Dahlia; Kuhn, Fabian; Abraham, Ittai; Balakrishnan, Mahesh; Gupta, Archit; Akella, Aditya. "A Unified Network "Coordinate" System for Bandwidth and Latency". Microsoft Research. https://www.microsoft.com/en-us/research/wp-content/uploads/2008/09/tr-2008-124.pdf. 
  8. ↑ Stokkink, Quinten; Ileri, Can Umut; Epema, Dick; Pouwelse, Johan (2023-05-01). "Web3 Sybil avoidance using network latency" (in en). Computer Networks 227: 109701. doi:10.1016/j.comnet.2023.109701. ISSN 1389-1286. 
  9. ↑ Chan‐Tin, Eric; Heorhiadi, Victor; Hopper, Nicholas; Kim, Yongdae (July 2015). "Hijacking the Vuze BitTorrent network: all your hop are belong to us" (in en). IET Information Security 9 (4): 203–208. doi:10.1049/iet-ifs.2014.0337. ISSN 1751-8717. 
  10. ↑ 10.0 10.1 Ledlie, Johnathan; Gardner, Paul; Seltzer, Margo. "Network Coordinates in the Wild". USENIX Symposium on Networked Systems Design & Implementation (4): 299–311. https://people.csail.mit.edu/ledlie/papers/wild07-tr.pdf. 
  11. ↑ 11.0 11.1 Ng, T.S.E.; Zhang, Hui (June 2002). "Predicting Internet network distance with coordinates-based approaches". Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies. 1. pp. 170–179 vol.1. doi:10.1109/INFCOM.2002.1019258. ISBN 0-7803-7476-2. https://ieeexplore.ieee.org/document/1019258. 
  12. ↑ 12.0 12.1 Costa, M.; Castro, M.; Rowstron, R.; Key, P. (March 2004). "PIC: Practical Internet coordinates for distance estimation". 24th International Conference on Distributed Computing Systems, 2004. Proceedings.. pp. 178–187. doi:10.1109/ICDCS.2004.1281582. ISBN 0-7695-2086-3. https://ieeexplore.ieee.org/document/1281582. 
  13. ↑ 13.0 13.1 Dabek, Frank; Cox, Russ; Kaashoek, Frans; Morris, Robert (2004-08-30). "Vivaldi: a decentralized network coordinate system". ACM SIGCOMM Computer Communication Review 34 (4): 15–26. doi:10.1145/1030194.1015471. ISSN 0146-4833. https://doi.org/10.1145/1030194.1015471. 
  14. ↑ Y. Chen et al. (April 2009). "Pharos: Accurate and Decentralised Network Coordinate System". IET Communications 3 (4): 539–548. doi:10.1049/iet-com.2008.0187. http://www.cs.duke.edu/~ychen/papers/IET_Pharos.pdf. Retrieved 2013-11-27. 
  15. ↑ 15.0 15.1 15.2 15.3 Liao, Yongjun; Du, Wei; Geurts, Pierre; Leduc, Guy (2013-10-01). "DMFSGD: a decentralized matrix factorization algorithm for network distance prediction". IEEE/ACM Transactions on Networking 21 (5): 1511–1524. doi:10.1109/TNET.2012.2228881. ISSN 1063-6692. https://doi.org/10.1109/TNET.2012.2228881. 
  16. ↑ 16.0 16.1 Mao, Yun; Saul, Lawrence K.; Smith, Jonathan M. (December 2006). "IDES: An Internet Distance Estimation Service for Large Networks". IEEE Journal on Selected Areas in Communications 24 (12): 2273–2284. doi:10.1109/JSAC.2006.884026. ISSN 1558-0008. https://ieeexplore.ieee.org/document/4016146. 
  17. ↑ Chen, Yang; Wang, Xiao; Shi, Cong; Lua, Eng Keong; Fu, Xiaoming; Deng, Beixing; Li, Xing (December 2011). "Phoenix: A Weight-Based Network Coordinate System Using Matrix Factorization". IEEE Transactions on Network and Service Management 8 (4): 334–347. doi:10.1109/TNSM.2011.110911.100079. ISSN 1932-4537. https://ieeexplore.ieee.org/document/6092405. 
  18. ↑ Huang, Haojun; Li, Li; Min, Geyong; Miao, Wang; Zhu, Yingying; Zhao, Yangming (November 2022). "TNDP: Tensor-Based Network Distance Prediction With Confidence Intervals". IEEE Transactions on Services Computing 15 (6): 3554–3565. doi:10.1109/TSC.2021.3089241. ISSN 1939-1374. https://ieeexplore.ieee.org/document/9454377. 
  19. ↑ Deng, Lei; Zheng, Haifeng; Liu, Xiao-Yang; Feng, Xinxin; Chen, Zhizhang David (2020-12-15). "Network Latency Estimation With Leverage Sampling for Personal Devices: An Adaptive Tensor Completion Approach". IEEE/ACM Transactions on Networking 28 (6): 2797–2808. doi:10.1109/TNET.2020.3022757. ISSN 1063-6692. 
  20. ↑ Madhyastha, Harsha V.; Isdal, Tomas; Piatek, Michael; Dixon, Colin; Anderson, Thomas; Krishnamurthy, Arvind; Venkataramani, Arun (2006-11-06). "iPlane: an information plane for distributed services". Proceedings of the 7th Symposium on Operating Systems Design and Implementation. OSDI '06 (USA: USENIX Association): 367–380. ISBN 978-1-931971-47-8. https://dl.acm.org/doi/10.5555/1298455.1298490. 
  21. ↑ Madhyastha, Harsha V.; Katz-Bassett, Ethan; Anderson, Thomas; Krishnamurthy, Arvind; Venkataramani, Arun (2009-04-22). "iPlane Nano: path prediction for peer-to-peer applications". Proceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation. NSDI'09 (USA: USENIX Association): 137–152. https://dl.acm.org/doi/10.5555/1558977.1558987. 
  22. ↑ Chan‐Tin, Eric; Heorhiadi, Victor; Hopper, Nicholas; Kim, Yongdae (July 2015). "Hijacking the Vuze BitTorrent network: all your hop are belong to us" (in en). IET Information Security 9 (4): 203–208. doi:10.1049/iet-ifs.2014.0337. ISSN 1751-8717. 



Retrieved from "https://handwiki.org/wiki/index.php?title=Network_Coordinate_System&oldid=3369067"