AFCOMQuarterlySUMMER2020_lowres (2)

THE AFCOM JOURNAL Trends and Analysis for the Data Center Professional AFCOM.com NNEEXTWT-OLERVKESL A TURBO BOOST FOR DATA CENTERS NEXT-LEVEL NETWORKING NEEDS: HOW THE INTERNET OF THINGS IS WHAT SHOULD 5G MEAN FOR SPEED, SPEED, SPEED TRANSFORMING NETWORKING DATA CENTERS? Data center network operators have compiled The way data is generated, gathered, and We know 5G can improve connectivity. a laundry list of must-have features they deem analyzed is fast becoming unfeasible to Now comes determining what connectivity essential for running their next-level networks. transmit everything to a central repository. can do for data centers, and what to do next Learn the results of this Omdia survey … page 6 Now what? … page 11 and what to do next to take advantage of the technology. … page 19

BRANDS PAGE SAVE THE DATE! DATA CENTER WORLD MARCH 8-11, 2021 ORLANDO, FL STAY CONNECTED @DataCenterWorld please recycle 20

Editorial From the Desk of Alicia Schap MEGHAN REILLY Group Director, IT Portfolio MAKING A SUCCESSFUL TRANSITION [email protected] TO OUR VIRTUAL REALITY KELLY KOCHER A s I approach my one-year anniversary since I joined AFCOM as association manager, I Managing Director can certainly attest, as we all can, much of the past year has not been anything I could [email protected] have anticipated. Bringing a background of process improvement and association management, with a strong focus on customer service, I hit the ground running. BRIAN GILLOOLY I analyzed our current systems and learned more about the needs of professionals in the data Content Director, DCW and AFCOM center and IT infrastructure industry. There is no amount of foresight or planning that could’ve predicted what we would all be experiencing just a few short months into my tenure with AFCOM. [email protected] But what I have discovered since our collective world has slowed down, socially distanced, KAREN RICCIO and isolated from one another, is that our AFCOM community is resilient, forward thinking and Content Editor, AFCOM innovative. These are all important traits to achieve success in the data center industry. And [email protected] through it all, AFCOM and our Data Center World team have been able to deliver up-to-date and relevant content that our members and industry need to survive and emerge as a stronger ALICIA SCHAP community despite unprecedented challenges. Association Manager [email protected] In April, with our Data Center World conference postponed until August, we delivered our State of the Data Center keynote in a virtual session with industry expert Bill Kleyman. This virtual format was SUSAN GILLESPIE not only informative but engaging and provided the foundation for our next foray into the virtual world. Account Manager/West [email protected] Our Data Center World team kept the momentum going in May and delivered our first virtual conference – Business Continuity & Best Practices – allowing our members and data center community COURTNEY WOOFTER to experience live sessions from our AFCOM experts to learn how to pivot and adapt to meet Account Manager/East the challenging and ever-changing demands impacting their data center operations. It was also [email protected] a first experience for networking and meeting with service providers in a virtual environment. VERONICA GOLDWEBER While we were all hoping to be meeting together this month in San Antonio, our virtual conference Marketing Manager, Data Center World experience gave us some great insight to switch gears again and take Data Center World fully virtual. I am excited to navigate through the immersive platform and engage with many more of you. [email protected] With all the virtual activity happening with Data Center World and AFCOM National, our AFCOM Corporate local chapter communities have also been working hard by hosting their own virtual events and also MARCO PARDI providing great educational presentations of issues impacting their local and broader data center President, BusinessTech Events, Technology and IT communities. I look forward to continuing to work with our chapters to find innovative ways [email protected] to stay connected with their local members and our broader AFCOM chapter network. RUSSELL BACON While we have all been navigating this ‘new normal,’ the one thing that has become more and Global Commercial Director, Telecoms more apparent to me is the resiliency of our members, our chapters, and our association. It’s no wonder that AFCOM has been the leading association for data center and IT professionals [email protected] for more than 40 years. And we have a lot more yet to come. Published by Informa Exhibitions LLC Looking at the months ahead, we’re going digital with all four of this year’s AFCOM Leaders 2020 N. Central Ave., Suite 400 Labs. If you are not familiar with these workshops, I encourage you to go to AFCOM.com and Phoenix, AZ 85004 click on the “Leaders Labs” tab to learn more about these exclusive gatherings of hand-selected +1 480 990 1101 thought leaders collaborating to further drive innovation, education and advancement of data Web: AFCOM.com center technologies – and how you can be a part of it. Our space is limited for each date and topic, so if you’re interested in participating, you’ll need to submit your interest form soon. We are also hard at work behind the scenes to provide additional AFCOM member benefits by developing a market research database, building additional partnerships and creating a network between our local chapters to strengthen our collective AFCOM community. If you have any other ideas that will enhance your AFCOM member experience, please reach out and send me an email, which you will find below. I look forward to ‘meeting’ many more of you in the coming weeks at Data Center World, Leaders Labs workshops, and various AFCOM chapter events. With our firm AFCOM foundation, together we’ll continue to bridge our physical distance and bring our community together and stronger than ever. All my best, Alicia Schap Association Manager [email protected] www.AFCOM.com 3

Contents AFCOM Vol. 2, No. 2 | Summer AFCOM.com 6 19 WHAT SHOULD 5G MEAN FOR DATA Getty Images CENTERS? How will 5G improve data center connectivity, and what kinds of applications will it enable? Learn specifically what 5G connectivity will mean for you, and what to do next. 22 BEST PRACTICES: YOUR EMERGING 5G JOURNEY CHECKLIST 6 NEXT-LEVEL NETWORKING NEEDS: Here is a checklist to go through before embarking on your 5G journey. SPEED, SPEED, SPEED 24 RENEWING COLOCATION CONTRACTS: GETTING MORE, SPENDING LESS Data center network operators have compiled Successfully renewing colocation contracts a laundry list of must-have features they deem results in getting more from your provider while essential for running their next-level networks, spending less. Learn strategies for doing so. according to the findings from a recent Omdia survey “Data Center Network Strategies and 28 PRACTICAL GUIDANCE FOR DATA Leadership.” CENTER MODELING Understanding the true data center financial 11 HOW THE INTERNET OF THINGS IS landscape enables accurate, fact-based TRANSFORMING NETWORKING decision-making. This article provides It is fast becoming unfeasible to transmit all step-by-step guidance and a practical the data generated, gathered, and analyzed approach to creating a data center cost model. into a central repository. How will data center managers stay on top of this, and what networking technologies are needed to deal with a massive influx of data? 14 CASE STUDY: IMPLEMENTING 28 DEDICATED BROADBAND AT THE EDGE UK-based Trunk Networks operates eight interconnected data centers dispersed around the country. But in this age of IoT, edge, and streaming services, that isn’t enough. Now what? 16 BEST PRACTICES FOR FUTURE PROOFING DATA CENTER NETWORKS Here are tips on how best to adjust networks to cope with the edge computing and IoT trends. Hint: Start small. Copyright © 2020 Informa Exhibitions LLC. All rights reserved. The publisher reserves the right to accept or reject any advertising or editorial material. Advertisers, and/or their agents, assume the responsibility for all content of published advertisements and assume responsibility for any claims against the publisher based on the advertisement. Editorial contributors assume responsibility for their published works and assume responsibility for any claims against the publisher based on the published work. Editorial content may not necessarily reflect the views of the publisher. Materials contained on this site may not be reproduced, modified, distributed, republished or hosted (either directly or by linking) without our prior written permission. You may not alter or remove any trademark, copyright or other notice from copies of content. You may, however, download material from the site (one machine readable copy and one print copy per page) for your personal, noncommercial use only. We reserve all rights in and title to all material downloaded. All items submitted to The AFCOM Journal become the sole property of Informa Exhibitions LLC. 4 www.AFCOM.com

All Data Centers, All the Time. In-depth, daily coverage of the business and technology of data centers and enterprise IT. • Hybrid cloud • Edge computing • Colocation • Silicon innovation • Enterprise networks • Intelligent data center power and • Machine learning in cooling design and management the data center • Digital operations sustainability • Kubernetes • Data center investment trends Join our community today! www.DataCenterKnowledge.com www.AFCOM.com 5

NEXT-LEVEL SNPEETEWDO,RSKPIENEGD,NSEPEEDESD: As all the ways organizations use data change, so do technical requirements of data center networks. By Omdia Research B esides sheer capacity demands, Getty Images modern networks consist of diverse sets of equipment enterprises about the features they most tracking, and support for AI: and software. They are more want in their data center networking Increased port speeds—a no-brainer distributed, reaching to edge technology today as part of its Data locations. And, coupled with the growth Center Compute Strategies & Leadership North upgrade decision that is vital for enterprises of machine learning applications, networks American Enterprise Survey – 2019. Based on evolving their networks to remain competitive have become more complex. the responses, they compiled a customer and current. wish list of sorts. Omdia analysts recently asked IT Interoperability—using more open, leaders from 140 North American At the top of the list, predictably, was standards-based equipment is becoming port speed, followed by interoperability, increasingly crucial for users as their application awareness, and VM movement networks become more diverse. This trend aligns with the increased adoption of Open 6 www.AFCOM.com

Compute Project (OCP)-certified products. Figure 1: DC servers (n=140, 140) Getty Images Application awareness and automated 2,500 2009 tracking of virtual machine (VM) Average number of servers 2,000 movements —these elements are important as more data center traffic gets routed 1,500 from central DCs to edge locations, and networking users need to monitor their data 1,000 1004 across compute and storage, independent 500 of the physical data center location. 0 2021 Support for AI—respondents are keen No w on the benefits AI promises, and have high aspirations for the technology to improve © 2020 Omdia the way their networks handle increasing amounts of resource-intense traffic from solutions. An example is the increased use Additionally, our respondents’ feedback machine learning (ML) and deep learning of open compute and bare metal switches, aligns with other Data Center Compute (DL) applications. which has driven more adoption of third- Strategies Survey results which show that party merchant, and programmable, data between 2019 and 2021, respondents Data center operators have spoken—and plane forwarding and CPU silicon. We expect a 53% increase in the average they say they want their vendors to offer a expect this trend to continue with more number of physical servers in their DCs. range of enhancements, including increased standards-based solutions emerging Galabov attributes this growth to new port speeds, greater interoperability, in response. applications like artificial intelligence (AI) application awareness, and support for and machine learning (ML), which require artificial intelligence (AI),” said Devan Adams, The requirement for applications to be more computing power to process larger principal analyst at Omdia. delivered on demand, coupled with the data sets, and enterprises’ plans to increase introduction of virtualization and data center the use of customer data for improved Here is a deeper dive into the findings of orchestration technology, has kicked off an business decision-making. Omdia’s Data Center Compute Strategies unprecedented transformation that began & Leadership North American Enterprise on servers and is now reaching into the data Data Center Sites: Continue to Survey – 2019. center network. Rise to the Challenge Introduction Significant Expansion Planned for Between 2019 and 2021, respondents DC On-Premises Infrastructure continued to invest in their DCs and are Application architectures continue to planning to more than double (nearly evolve with the increased adoption of Between 2019 and 2021, respondents triple) their average number of DC sites. software containers and micro-services indicated they expect to more than double Expansion of respondents’ DC sites can be coupled with a Dev/Ops culture of rapid the average number of servers in their DCs. attributed to several factors, including many and frequent software builds. In addition, This is a noticeable rise from the 2019 results smaller DC sites (<10 servers) increasing we continue to see new technologies in which respondents expected a 35% to 10+ by 2021, an increasing average such as artificial intelligence (AI) and rise from 2018 to 2019 and is consistent number of servers deployed (as stated in machine learning (ML) incorporated into with results from the Omdia Data Center the section above) at edge- located DCs applications. These applications consume Compute Strategies & Leadership North to satisfy lower-latency requirements, and network bandwidth in a very dynamic and American Enterprise Survey – 2019, in which local government regulations. unpredictable manner requiring increased 74% of respondents stated they plan to use of DC networking equipment. increase their server spending. Investments and Spending Expected to Rise Traditional methods for network Vladimir Galabov, principal analyst for provisioning to provide users with a Omdia’s Data Center Compute Research, The leading drivers for investment in data quality experience, such as statically attributes this server growth to data and center technology are reliability and security assigned priorities (QoS) in the data insight on data becoming business drivers in (79% of respondents). center network, are no longer suffice. the enterprise and reinforces his hypothesis The data center network must be able to that equilibrium will be established between Improved reliability and security automatically recognize application traffic servers deployed by enterprise, telco, and are the top drivers in 2020, signifying flows and rapidly adjust priority to match cloud service providers (CSPs). that enterprise respondents remain the dynamic nature of application traffic in a resource-constrained world. Newer software technologies like software-defined networking (SDN) continue to drive more open networking www.AFCOM.com 7

Figure 2: DC sites (n=140, 140)Average number of data centers 127 increase of investments in tools to improve automation and analytics. As in 2018, very 140 2021 few respondents expect to reduce or stop spending in the categories listed, which is 120 © 2020 Omdia good news for vendors. 100 Network Automation and Network automation and network Analytics Expect to See Highest analytics remained the categories most 80 Investment Increases respondents expect to increase investments in over the next two years; 69% and 60 Respondents provided details about 64% of respondents, respectively. As more workloads shift to the cloud with 40 44 their investment plans for data center IT use of hybrid cloud architectures, it is 20 increasingly imperative for networks to infrastructure over the next two years. adopt automation techniques. 0 No w In the next two years, respondents Respondents are expecting to increase spending on physical switches over the focused on ensuring their networks showed that their data center investments next two years; this pertains to both Fibre run seamlessly as workloads increase Channel (FC) and Ethernet switches: and application use expands, without on networking (hardware and software) sacrificing their commitment to keeping Respondents (62%) said they plan to users and company data safe and a high will continue. Like in 2018, spending on enhance spending on FC switches, which priority. As application use continues is somewhat contradictory to results from to soar, putting more strain on data virtual outweighed those on hardware the latest Omdia Storage Area Network center infrastructure, enterprises must Equipment Market Tracker - Q3 2019 where also deal with rising security risks from devices, with respondents prioritizing the FC switching revenue is only expected to brute-force distributed denial of service grow by 2% year-over-year (YoY) in 2020, (DDoS) attacks to advanced threats and is expected to decline by 1% YoY in and malware, making the process of 2021. operating a data center more challenging to their IT teams. The main contributors to FC switch market stability over the next few years will be Improved scalability, automation and 32G/64G FC switches and the integration of application performance, plus rapid applications deployment, were the next key Figure 3: DC investment drivers (n=140) 79% drivers that respondents chose. Scalability is becoming more critical to enterprise Improve reliability 79% respondents as the amount of big data use expands, putting more strain on core Improve security 74% data center infrastructure as processing demands rise. Improve scalability 71% Much of the big data demands stem Enable automationDrivers 71% from increased use of compute-heavy Improve application applications like ML, AI, and deep learning, 69% and will also require enterprises to place a per for man ce higher priority on the speed and performance Deploy applications 66% of their applications. As DC networks and applications become more complex and rap idly 66% automation tools increase, enterprises need Enable programmable solutions that give them better control of 64% their networks and the variety of applications data planes that run on them without requiring significant Simplify operations 64% manual intervention. Data center consolidation 57% Improve management capabi lit y 20% 40% 60% 80% Decrease costs Percent of respondents rating 6 or 7 © 2020 Omdia 0% 8 www.AFCOM.com

Figure 4: Weighted Average of Switch Ports (accepted and inspired) over the next two. Application awareness and automated Weighted average of switch ports 100% 8% 5% 80% 46% 8% virtual machine (VM) movement are 60% 10% becoming more important as more data 40% 8% CAT6 copper Direct attach 11% CAT5 copper center traffic gets routed from central DCs 20% 11% 18% to edge locations globally. The results show 0% 18% 47% © 2020 Omdia that data center networks can no longer operate in a silo and respondents want Single-mode fiber 9% 2021 solutions that are easy to integrate, allowing No w CAT7 copper them to monitor their data across compute Multi-mode fiber and storage, independent of physical data center location. new technologies like non-volatile memory a key functionality to meet the requirements express over FC (NVMe-over-FC) and storage of buyers looking to bring automation to AI and Network Virtualization area networking (SAN) telemetry. Based on their data centers. Deemed Key for Decision-Makers the Omedia analysis, it does not appear that the FC switch market is dead as many have Therefore, the results below are no AI support (64% of respondents) is a speculated, yet there are signs that its best surprise with the shift to the cloud new feature added in the 2019 survey. years are behind it and the total addressable continuing, as enterprises indicated they Network virtualization protocol support market (TAM) is shrinking, which is why we expect to spend on average 28% of their IT is a long-standing option; its ranking rose forecast the CY18–23 revenue compound budgets on off-premises cloud services in to a tie at fourth in the 2019 survey versus annual growth rate (CAGR) to be -4%. 2019, up from 22% in 2018, where vADCs fifth in 2018. are most prevalent. The 2019 survey shows Of the total respondents, 61% said they that more respondents expect to keep their Respondents seem to be keen on plan to increase spending on Ethernet data center spending flat for ADC hardware the benefits that AI promises and have switches (physical). The biggest contributors (41%) versus on vADCs (38%); these high aspirations for it to be able to bring to the growth in the proposed 2020–21 results are anticipated given the trends we realistic improvements to their data center timeframe are bare-metal switches, with are seeing as hardware revenues shift more networks. Many vendors claim their high- revenues expected to grow at 55% and to software in many parts of networking. throughput (100GE+) data center switches 28%, respectively. are designed for use in environments Respondents Want it All, Higher required to handle resource-intense Of all respondents, 58% (significantly Speeds and More applications like AI and ML; yet, Huawei higher from 42% in 2018) plan to took it a step further by introducing an AI increase spending over the next two In 2019, 65% of respondents rated speed chip embedded in its CloudEngine 16800 years on software-defined wide area highly on their list (similar to 2018 where it series switches, which it says supports networks (SD- WANs). This aligns with ranked first), openness and interoperability 400GE ports and iLossless algorithms to the ongoing shifts to hybrid cloud and (a vast improvement from 2018 where it autosense and auto-optimize traffic. multi-cloud architectures for off-premises ranked tenth) and application awareness cloud services and increasing use of (significantly better than 2018 where it It remains to be seen how AI features software-defined networking (SDN) for ranked ninth) as key reasons to invest in will impact the data center networking the automation of wide area network data center networks. market and what qualitative and quantitative (WAN) deployments. outcomes will result. Increased port speeds are an obvious Respondents (51%) plan to increase upgrade decision, and the results show that Omdia is a global technology research spending in the next two years on virtual it remains top of the mind for enterprises; powerhouse, established following the analog-to-digital converters (vADCs), therefore, vendors need to take note merger of the research division of Informa and 48% plan to increase spending for that these enhancements remain key to Tech (Ovum, Heavy Reading and Tractica) physical ADCs. Virtualized ADCs are customer’s networks. and the acquired IHS Markit technology primarily used in cloud-architected DCs, research portfolio. Omdia combines the permitting the capability to integrate with In alignment with the promising openness expertise of more than 400 analysts across DC orchestration software and providing results, respondents also expect to increase the entire technology spectrum, analyzing visibility on application traffic performance, their use of OCP-certified switches 150 markets publishing 3,000 research solutions, reaching over 14,000 subscribers, and covering thousands of technology, media & telecommunications companies. www.AFCOM.com 9

CONTENT ADVISORY TEAM 2020 WE’VE CREATED AN OUTSTANDING PROGRAM FOR YOU FOR DATA CENTER WORLD 2020 — A VIRTUAL EXPERIENCE! You can rest assured that the lineup of speakers and subject matter is the best the industry has to offer! And that’s because of the dedication and passion of our 2020 Content Advisory Team, a group of devoted professionals whose experience and insight runs the gamut of the data center industry. We’re gratified to have been given the trust of these busy individuals to evaluate, debate, and select the finest content for this year’s show. We extend our thanks to this amazing crew! Stephen Barbour Steve Bass Janna Connor Tad Davies Hector Diaz San Antonio Data Center Global Data Center Unit Manager Data Centers, President, Managing Director, Mission Manager, Lowe’s Manager, Chevron Health Care Service Fodere Critical Sector, Servtech Inc. Corporation Christine Ficke Brian Gillooly Carrie Goetz Kevin Kent Lori Murphy Manager, National Key Part- Content Director, Founder & CTO, Data Center Operations Man- Senior Account Manager, ners, CommScope Data Center World StrategITcom ager, OSU Wexner Medical DataSpan Center John Parker Nelson Sigoloff Yevgeniy Sverdlik Global Data Center Opera- Data Center Engineering Editor-in-Chief, Data Center tions and Disaster Recovery, Architect, Valero Knowledge Esri

HOW THE INTERNET OF THINGS IS TRANSFORMING NETWORKING With the way data is generated, gathered, and analyzed, it is fast becoming unfeasible to transmit everything to a central repository. Efforts to do so clog networks and slow operations. Edge computing provides a solution to this problem. But how are data center managers going to stay on top of this, and what networking technologies do they need to deploy? By Drew Robb T he traditional data center networking model has been under attack for some time. Instead of all data being housed under one roof, the allure of cheap storage and abundant compute power without the hassle of management, upkeep, or regular refreshes has pushed a lot of data onto the cloud. Similarly, Software-as-a-Service (SaaS) applications deployed by lines of business further dispersed the network. With the rise of Internet of Things (IOT) and the growth of edge computing, the data center network is going to shift radically once again. Greater complexity has entered into data center infrastructure as the cloud, edge computing “Edge and IoT deployments are stretching and the IoT grow. infrastructures, shifting priorities, and adding constricted by latency more than bandwidth. In addition, evolving complexity,” says David Cappuccio, an analyst at Gartner. applications, such as self-driving/connected cars, the latest smart A tipping point is near where the volume of traffic coming into the phones, augmented reality (AR), virtual reality (VR), robotics, gaming, enterprise data center network becomes greater than the quantity some medical applications, and drone technology can’t tolerate the going outward. And it’s only going to get worse. According to latency inherent in round trips from the edge to the core. International Data Corp. (IDC), 55% of all data will be generated by Where the traffic volume merits, therefore, edge data centers as many as 80 billion IoT devices by 2025, and 70% of enterprises are emerging to take some of the strain off networks and major will run varying levels of data processing at the IoT edge by 2023. data centers. But organizations are cautioned to ensure there is a In many cases, the bulk of IoT and artificial intelligence (AI) traffic value proposition in place before investing in edge infrastructure. will be generated and processed at the edge. Although it is much cheaper than data center gear, it is pricier than “There is more data being generated, processed, initially analyzed, typical edge of network equipment. After all, it is no longer just a and used at the edge due to the IoT and other trends such as relay point. It contains compute power, storage capacity, and even streaming media,” says Greg Schulz, a consultant at StorageIO some analytics. Group. “Not all data from the edge needs to flow back to the “Edge data centers will be imperative to achieving the low- core: Some can be batched up and sent via hybrid appliances latency, real-time communication needed to support emerging (e.g. physically ship) or summarized and sent to the cloud or data 5G technologies like autonomous cars, cellular IoT, and smart center over the network.” cities,” says Telecommunications Industry Association (TIA) CEO The Rise of the Edge Data Center David Stehlin. 5G is yet another contributor to edge congestion. According to The use cases for edge computing are many and varied. the Ericsson Mobility Report, 5G mobile network subscriptions Content streaming and interactive entertainment, for example, are will exceed 190 million by the end of 2020 and 2.8 billion by 2023, www.AFCOM.com 11

accounting for more than 20% of mobile 5G extends networking capabilities into the 1 Gbps and beyond range and can take latency down below 10ms. data traffic. In the very near future then, it will become all but impossible for many be delegated to the edge based on pre- rise exponentially, the cost equation moves organizations to maintain the networking programmed rules. This reduces the data further in favor of edge data centers. model where all data is shipped to a central center workload by executing routine actions data center. Capacity challenges, bandwidth without human involvement. More major Joseph Campbell, Energy & Manufacturing bottlenecks, latency: The cost of addressing decisions, would be referred to central IT. CTO at IT and consulting firm T-Systems these challenges will skyrocket. (a subsidiary of Deutsche Telekom), calls A survey by Forrester Research found attention to the difference between high Edge computing, then, is all about placing IT that 56% of enterprises have adopted or are frequency and low frequency data streams. resources nearest to users, devices, sensors, expanding deployment of IoT solutions. A A door sensor sending a packet when it and equipment such as vehicles traveling on further 17% are piloting IoT programs. Yet is opened is low frequency and requires highways. Faster collection and analysis of data only 29% of organizations have expanded little bandwidth; latency of response means real-time or near real-time detection of their analytics strategies to include edge IoT. is not a concern.  However, acoustic trends, anomalies and opportunities. sensors attached to pumps moving large “It may not always be clear which use hydrocarbons or potentially explosive In the case of an autonomous car, the cases would benefit most from edge IoT for material require low latency and high compute power must be present around the analytics,” says Sophia Christakis, an analyst bandwidth. The goal of analytics at the edge highways. In busy areas, small edge data at Forrester Research. “Topping the list are is to seek out anomalies from mountains of centers will be required to receive, process, requirements for real-time response and repeating patterns and routine traffic. analyze, summarize, and transmit both impact on customers’ experiences — the centrally and to the vehicle. This data comes greater the importance of these factors for Edge-Enabling Technologies from myriad sources: Sensors in the vehicle, in any given use case, the greater the likelihood nearby or approaching vehicles and in the road that edge IoT makes sense.” Campbell believes 5G and software- itself; imaging systems; radar and LiDAR (light defined networking (SDN) can help mitigate detection and ranging); and GPS, to name a She notes that IoT insights are perishable. many IoT and edge challenges. few. Once received at the edge, it needs to Their value diminishes as time is added. be organized, analyzed, and fed back to the In some cases, only instant response has “SDN will be key as 5G will drive a massive vehicle. There is no room for latency when any value. But beyond those use cases increase in in-bound networking channels,” you are dealing with collision avoidance, blind demanding speed, some have found it to says Campbell.   spot monitoring, lane departure warnings, be more cost-effective to process the bulk and steering a car safely to its destination. of data at the edge than to send it to the 5G is the fifth generation of mobile wireless Therefore, networking configuration should cloud or to the data center. As data volumes systems. It is a way for either mobile or accommodate local interaction. stationary devices to send and receive data Analytics at the Edge Frost & Sullivan analyst Karthik Sundaram points out that the data analysis is much faster at the local device level. The alternative is transmitting all sensor and edge device data across the network to the cloud or data center via wireless, satellite link, or cabling. Either way, too much time is consumed in sending, processing, and interpreting data. As well as latency, this approach overloads centralized infrastructures and creates bottlenecks. Real-time analysis can only be achieved by edge processing. Edge devices sift out the important from the unimportant. They send on summarized information – the tiny fraction of the data that has real value. They implement analytics to automate basic tasks, send alerts to technicians, and keep management informed. Autonomous decision-making can 12 www.AFCOM.com

without a physical connection. Many of the “SDN will be key as 5G will drive a massive increase in evolving use cases for edge computing will in-bound networking channels.” —Joseph Campbell, T-Systems rely on 5G to transmit data from IoT devices. “A new way of thinking is needed where The solution in this case was not heavy As shown in the chart on page 12, existing the network encompasses the edge as well investment in edge data centers at each networks are reaching their limits when it as the center,” says Ruppin. location. Instead, it deployed Riverbed comes to video streaming, first responder Steelhead wide-area data services applications, wireless cloud-based offices, Ananda is building a network known as appliances to augment the corporate WAN. multi-person video calls, real-time gaming, secure global LAN (SG-LAN) with speed They reduced the company’s bandwidth and disaster alerts. As these fields develop and security built-in. This cloud-managed, utilization up to 80%. That dropped the further, they will require even more throughput private overlay network connects all edge contract approval process down to a few and/or lower latency. Already, functions like computing devices to the data center or minutes. This organization still favors autonomous driving, AR and VR lie beyond cloud services, as if they are on a LAN. centralization of IT over a more distributed the reach of existing networks. 5G is seen architecture. as the way to provide such services. “All edge computing devices operate on their own private, encrypted network that “Centralizing our IT operations reduces SDN is seen as a way to enable network is invisible to the outside world and has the number of servers, parts, and IT evolution. It’s a move away from hardware- no external attack surface,” says Ruppin. personnel needed to support the business,” focused infrastructure to create more “Devices find the optimal path to maximize says Rich Jackson, vice president of customized and agile networks. SDN speed and throughput and get around any technology operations at Hilton Grand separates the control plane from the data networking issues.” Vacations. “It allows us to maintain strong plane so both can be optimized based configuration management and change upon business needs. Network policies For those not wishing to invest in their own control standards and reduces the overall and applications are centrally programmed edge data center network, service providers, cost of the technology to provide higher and managed through a single controller and telecoms are a possible alternative. service levels.” instead of configuring physical devices CenturyLink is readying its own network one by one according to vendor protocols. for the edge, 5G, and IoT onslaught. It is Edge Taking Over? Benefits include simplified management, implementing SDN and policy automation greater network agility, lower costs, and while putting more compute power and Enhanced interconnectivity is about to improved security. SDN also facilitates easier intelligence at the edge that is easily enable improved edge access to more core edge computing and smoother interconnects accessible to its global fiber backbone. applications. With new IoT and other use between edge and core data centers. These co-located edge capabilities can be cases, edge infrastructure is poised to be turned up or down as traffic volumes dictate. one of the main growth engines in the server Switches and routers are already and storage market for the next decade and benefitting from SDN and SDN-like features. Similarly, Equinix has been building out beyond, according to IDC. But that doesn’t Aruba (a Hewlett Packard Enterprise its 5G and edge capabilities. It recently mean the traditional central data center company), for example, now offers a built-in announced the opening of an International is going anywhere. Gartner predicts that Network Analytics Engine in its CX switch. It Business Exchange (IBX) data center despite the rise of the cloud, the IoT, and utilizes machine learning-based technologies and the launch of its 5G and Edge Proof the edge, fewer than 10% of enterprises to continually ingest and analyze a wide of Concept Center (POCC) to support will decommission their in-house centralized variety of network metadata, which is growing demand to test and validate new data centers over the next two years. then provided to network administrators 5G and edge technologies. Data centers as additional insights about real-time can directly connect to its 5G and edge “Despite the rapid adoption of public performance and health information through sandbox environment to prove out complex cloud for new applications, few enterprises the CX management platform. deployment and interoperability scenarios. are actually decommissioning data center networks,” said Gartner analyst Andrew But even technology such as 5G, intelligent This is not to say that the edge is where the Lerner. “We see continued interest and switches, and SDN may not be enough to bulk of compute power is eventually going. It investment from most Gartner clients satisfy edge computing and IoT needs. Adi is all down to use cases. Take the example involved with data center networking Ruppin, founder & CEO of startup Ananda of Hilton Grand Vacations. Rapid growth led infrastructure.” Networks, is addressing the design of the to challenges for its distributed network of Internet and how it handles connectivity, offices in New York, Osaka and Tokyo, Japan; Drew Robb is a veteran technology writer speed, and security in edge-heavy Guam, and Orlando. Sales cycles were stalling based in Florida. environments. He believes 5G is no magic on contract approvals. Customers had to bullet for networking woes. A 5G network wait more than 30 minutes while deals were still has to connect with a great many other processed due to network latency. parts of the network that utilize older, slower and less secure technologies and protocols. www.AFCOM.com 13

CASE STUDY: IMBRPOLAEMDBENANTIDNAGTDTEHDEICEADTGEED By Drew Robb T runk Networks, based in the UK, operates eight Getty Images interconnected data centers dispersed around the country. But in this age of IoT, edge, and streaming services, that “We realized that with the demand for data increasing, and local access networks moving from copper to fiber, our 10 isn’t enough – particularly for the many gamers who utilize Gbps backbone wasn’t going to be good enough,” says Drew Marshall, director of Trunk Networks. “To ready ourselves for the services of Trunk. They demand low latency and reliable the next big jump in demand, we decided to implement edge computing.” connections that won’t be throttled back during peak traffic Trunk added SLX 9030 switches and SLX 9640/9540 routers periods. from Extreme Networks to its data centers. The company is in the midst of installing more of these devices at prime edge locations Data center management personnel realized that bottlenecks around the country. These 100 Gbps-capable switches and deep- and troubleshooting challenges would only multiply if they continued to rely on a centralized data center infrastructure. 14 www.AFCOM.com

buffer border routers provided the power, “Keep reassessing where the edge needs to be price point, and low rack density required and where your customers and workloads are.” to satisfy organizations and gamers that want ultra-low latency, says Marshall. —Drew Marshall, Trunk Networks Another benefit he mentioned is a and services provisioned rapidly. In some Vodafone is steadily introducing more IoT reduction in traffic being transmitted to cases, extra edge capabilities require only end points. Marshall realized that it is the various data centers. Marshall says a few minutes to establish. up to Trunk to look at the needs of such some traffic remains only at the edge while partners and help them by moving its data other traffic is sent up to the core. This Marshall’s advice to other data center closer to them. But this should also work architecture minimizes the round-trip time managers is to continually review where both ways. For example, Trunk asked one for users as well as the volume that has to the edge of the network is; he has seen partner if it could pre-process data. This be sent to the core. it change on a week-by-week basis. As turned out to be simple for the partner, yet customers are added, and more IoT and it saved a lot of time and effort for Trunk. “When you don’t need to transfer content other smart devices come online, demand from one end of the country to the other, grows to localized traffic and reduces “All networks are dealing with the strain you can provide a better experience to latency. of IoT at the edge,” says Marshall. “It the user,” says Marshall. “Management, requires a higher level of interaction and monitoring, and automation become much “Keep reassessing where the edge cooperation to make it work for all sides easier when you have the same equipment needs to be and where your customers to avoid having to deal with excessive at the core and at the edge.” and workloads are,” says Marshall. amounts of data flow if it can be avoided.” Previously, he says, monitoring was An important aspect of this concept Drew Robb is a veteran technology writer done centrally. If an incident happened of the evolving network edge is to based in Florida. like a Distributed Denial of Service (DDoS) consider your partners and where they attack or a fiber break, troubleshooters had are located. Trunk, for example, partners to rely on the data making it back to the with Vodafone, a UK cellular provider. core. That didn’t always happen. The new routers from Extreme Networks ensure that Getty Images data center managers can stay on top of networking issues wherever they may be around the country. These devices include a built-in third- party VM that can run a Linux server. There is enough computer power available as well as a 10-Gbps internal data path to run monitoring probes on the routers to and automate many of the mitigation efforts close to the edge. Data is extracted from the network to be viewed directly via analytics tools on the guest VM. This also permits IT to sample traffic locally. “Central monitoring used to bring high volumes of traffic to the core,” says Marshall. “We can now monitor things locally and rely on the VM to report up on anything problematic.” He estimates a drop of at least 10% in incoming traffic through the decentralization of monitoring traffic. Overall efficiency has been raised, and network capacity has been freed up. Templates can be used by data center personnel to build out new network nodes with the same VMs, applications, www.AFCOM.com 15

BEST PRACTICES Getty Images FOR FUTURE PROOFING DATA CENTER NETWORKS Here are tips for data center managers on how best to adjust their networks to cope with the edge computing and IoT trends. By Drew Robb F rost & Sullivan analyst Karthik Don’t forget security and training. down walls and other barriers to support Sundaram recommends starting Sundaram stressed the importance of smooth data movement from edge to core small when implementing edge training data center personnel on edge and beyond. A territorial attitude to data computing. Prove its value first, and IoT technologies. On the security could get in the way of network evolution. then scale it up. side, securely configure device interfaces Software-defined networking (SDN) He also advocates investing in edge with the right level of access control and technologies are top of mind for Drew physical security. infrastructure now instead of sticking Greg Schulz, a consultant with StorageIO Schulke, vice president of Networking, Dell to traditional architectures that may Group, advises data center designers to Technologies. He says to adopt modern SDN become obsolete soon due to lack of IoT work with network, server, storage, and architectures and technologies that provide functionality. other data infrastructure designers to break better scalability, performance, monitoring, 16 www.AFCOM.com

For the edge, location is dictated by service latency make up the network, choosing friendlier requirements and access to network resources. technologies, and looking at the problem holistically.   automation, security, and management (high power PoE), along with wireless capabilities. In doing so, networks become technologies like Wi-Fi 6, Bluetooth and Consider colocation or the cloud, says smarter and more resilient. Zigbee. They each provide the capabilities Gartner analyst David Cappuccio, as a way and future proofing to support the influx of to supplement the data center network To solve the puzzle of how to gain the IoT devices. in order to exploit the interconnection most value from existing assets while services of provider networks. Connections introducing edge, IoT and SDN-enabled Dave Shacochis, vice president of may be as simple as fiber-optic cross- technologies, Schulke suggests partnering Enterprise Technology and Field CTO at connects but allow data center-based with IT OEMs, telecoms, cloud providers, CenturyLink, finds it wise to spend time assets to horizontally connect to multiple MSPs, and ISPs who know the landscape figuring out which way the data and traffic carriers, cloud providers, peers, and and can advise accordingly. flows will go in order to isolate potential service providers. This can create a choke points. Based on where IoT sensors more agile, dynamic infrastructure that Additionally, Schulke tasked data center and smart devices are located, determine can be adapted to rapidly as business managers with getting involved early in the where edge computing can add the most requirements change. planning stages of an IoT implementation value while reducing latency or network to ensure the infrastructure is in place stress, he says. All workloads are not equal, and proper to support the influx of devices and to placement is key to unlocking their true proper security mechanisms and tools To avoid device sprawl and minimize potential to the business, says Cappuccio. are in place. security challenges, Shacochis recommended keeping the hardware The Telecommunications Industry According to Gartner analyst Andrew and software at the edge consistent with Association emphasizes location. Lerner, data centers must institute better that of the core data center. Further, this Traditional data centers are often selected automation, orchestration, integration streamlines the software delivery pipeline due to geographic considerations, power and consistency for edge and core data for both the edge and the center. availability/cost, customer demand, real- centers to work in harmony. estate price, and physical/security risk Keep it simple, advises Adi Ruppin, factors. For the edge, however, location is CLI-based management tools Founder & CEO of Ananda Networks. dictated by service latency requirements for configuration, monitoring, and Reduce the number of solutions that and access to network resources. troubleshooting have to go, says William Choe, vice president of Product Line Drew Robb is a veteran technology writer Management for Campus Networking based in Florida. at Aruba, a Hewlett Packard Enterprise company. These tools are unable to Getty Images accurately provide information for all network-connected devices, which is a security risk. New networking technology such as the Aruba CX operating system replaces CLI commands with common English terms to simplify activities such as MAC requests, and drilling down to switch ports to resolve performance, setup, and configuration issues. Choe emphasizes an application-centric approach. Place workloads where they are best suited for given applications. Whether at the edge, in the cloud, or in a traditional data center, keep architectures simple, as flat as possible for higher performance, and focus on reliability over complexity. For IoT connectivity, Choe recommends wired technologies like IEEE 802.3bz (multi-gigabit) and IEEE 802.3af www.AFCOM.com 17

AFCOM ADVISORY BOARD Geoffrey Cope Michael Jamie Fogal Carrie Goetz Cindy Joos Cunningham Senior Vice President, Infrastructure and Founder & CTO, Former VP, EDG2 Director, University Data Operations Team Leader, StrategITcom Global Shared Services Centers, The University of Quicken Loans Cyxtera Mark Schultz Texas at Austin The AFCOM Advisory Board is a group of industry experts who Senior Project Manager, Ben Stewart collaborate directly with AFCOM management to provide guidance JE Dunn Construction and feedback on value propositions for the AFCOM membership. VP Engineering and Specifically, the board’s responsibilities include providing expertise Innovation, Cyxtera beyond AFCOM’s current management; knowledge, understanding, and strategic advice to AFCOM management to provide balance in advancing goals; a sharp focus on how to improve AFCOM and collectively set a direction to increase membership and value for its members.

WHFOATRSDHAOTAULCDEN5TGEMRSE?AN By Danny Bradbury 5 G technology is here … kind of. Getty Images Wireless carriers are in the early senior analyst for carriers and enterprise deployment stage, but the 5g networking at Moor Insights & Strategy. devices at the edge. In 5G parlance, this is infrastructure that exists today is Carriers are upgrading their base stations; known as 5G New Radio (NR). just a precursor to what’s coming. but the core of the network, which handles It’s the first generation of wireless functions like subscriber data management, Whereas LTE focused entirely on lower- mobile technology that promises to alter is still tethered to LTE. This model, known frequency radio spectrum, 5G spans low, wireless communications networks as well as non-standalone, supports LTE-based mid, and high bandwidths. 5G operators as data center infrastructures overall. In this communications along with some 5G are especially hungry for low- to medium- article, we outline what it is, what it means functionality, and he calls it ‘5G 1.0’. frequency spectrum because they offer for data center operators, and how they can longer-range communications. Conversely, prepare for it. “Eventually, those operators will upgrade high-frequency radio communications are 5G isn’t so much a single technology the core to 5G and get to a standalone limited to smaller distances, potentially as it is a collection of them, all designed model, which delivers all the promises of the increasing the investment required for to improve last-mile radio connectivity. It’s sub five millisecond latency and throughput denser coverage. Investment costs in a departure from the Long-Term Evolution that’s ten times that of LTE today,” he says. the 2GHz window would mirror those of (LTE) standard that defined 4G. Benefits He refers to that model, in which everything traditional LTE networks, according to include higher bandwidth, according to the will be 5G-native from the core to the user consulting company McKinsey, requiring International Telecommunications Union’s device, as 5G 3.0. between two and five sites per square minimum requirements as outlined in 2017. kilometer in densely populated areas. At This defines peak downlinks of 20 Gbit/sec What, then, is 5G 2.0? That phase will higher frequencies, carriers are looking at and peak uplinks of 10Gbit/sec, although focus on deploying more of the infrastructure 15-20 sites in the same area. that’s for the entire mobile cell. User data that data center operators and 5G users rates should hit 100Mbit/sec and 50Mbit/sec care about: the base station and radio When low-to-mid spectrum opportunities respectively, the document says. It should infrastructure that communicates with 5G come up, interest is consequently high. For also introduce lower latency, ultimately achieving at least 4ms. The Long Road to 5G That’s the promise of 5G, but getting there is an expensive and time-consuming process, requiring a substantial investment on multiple fronts. Carriers have to upgrade the base stations that connect to phones and other devices, and they have to license new radio spectrum to support the faster communications with more devices that is a key part of the 5G proposition. The first wave of 5G only gets us part of the way there, explains Will Townsend, www.AFCOM.com 19

example, in February 2020, the Citizen Band Getty Images Radio Services (CBRS) Alliance completed specifications for incorporating 3.5GHz CBRS into commercial 5G usage. This is the mid-range radio band traditionally used for CB radio, and while much bandwidth remains unlicensed, the FCC began auctioning off the licensed part on July 23, 2020. This is significant not only for carriers, but also for enterprises large enough to license spectrum and build their own private 5G network infrastructure to handle bespoke applications. In Germany, several large companies including Bosch and Volkswagen plan on doing just that using spectrum on offer there. Putting 5G to Work 5G’s Effect on Data Centers building data centers with the computing density to support that intensive work. What kinds of workloads will 5G support? These applications will affect where data Data center designers must accommodate For many consumers, it’ll be more of the centers are located, what goes in them, and increased power density and cooling, same: faster Netflix streaming and cat how they connect to the network, explain and GPUs. NVIDIA is already shipping videos, but there will be newer applications experts. Some will require either fast data GPU compute models designed for edge like augmented reality and virtual reality. processing at the edge for data delivered environments like these. Gaming applications keep coming up in via 5G, fast transmission of data to a remote conversation, too. location, or both. It’s creating what Digital Transforming Connectivity Realty’s CTO Chris Sharp refers to as “data There will also be a range of applications gravity”; a redistribution of data along a 5G will also change the nature of data on the enterprise side, say experts. In a continuum from the core of an enterprise’s center connectivity. Many edge access data manufacturing facility, putting computing IT infrastructure to the edge of the network centers will either be managed by telcos power closer to the edge could help and at various places in between, depending themselves or will peer with their facilities, a company to visually sort items on a on the needs of the application. explains Nitin Rao, senior vice-president of conveyor belt or monitor vibrations in global infrastructure at Cloudflare. Telcos used factory equipment. Many companies are preparing for this to peering with companies at one central transition. In a 2019 IT survey of more than 750 site in a city must open up hundreds or even Another application is enhanced robotics, organizations sponsored by Digital Realty, 451 thousands of breakout points where they can says Jacques Fluet, director of the Network Research saw 51% of respondents planning connect with clients’ data centers at the edge. Service Assurance Program at the to implement IoT in at least 11 locations in Telecommunications Industry Association metro areas within the next year. Almost one They must also be able to send that traffic (TIA). Robots have traditionally been fixed in every ten were planning to roll out more directly to other peering points nearby to the floor, but now manufacturers want than 50 such locations. rather than hauling it back to a distance them to be mobile. They must coordinate core location, he warns: “Have they actually with each other to do that effectively, which Companies are already exploring micro redesigned the way peering works to be means on-machine sensors communicating data centers. In June, Digital Realty signed able to peer not in one location in Chicago, with local edge computing power via reliable an agreement with Vapor.io to roll out but in five or 25 or 100?” high-speed local wireless networking. facilities in Atlanta, Chicago, and Dallas. They can run edge-native applications either The mechanics of network peering That collaboration will also extend to at the access edge or at Digital Realty’s are undergoing a radical change. In LTE production quality control, he explains: regional data centers. networks, radio access networks (RANs) “If a robot finds a fault in a weld inside a at base stations control the flow and car on the production chain, they will redo Sharp also expects AI-driven applications processing of cellular data between the cell the welding by slowing down the whole to change what’s in the data center. AI tower and the local mobile switching center chain and letting the other robots know training and inference for everything from (MSC), which routes data and calls across that they’re going to need an additional autonomous vehicles to human behavior the cellular network. Each RAN includes five seconds.” analysis will take some heavy lifting close the radio itself and a base band unit (BBU). to the network’s edge, he says. That means 20 www.AFCOM.com

This is a proprietary specialist hardware unit Getty Images that processes and routes the data passing over the radio. founder at analyst company Rethink Research. Jockeying for Position One of them is splitting out that BBN Putting a BBU at each cell site is expensive, functionality without affecting performance. There’s a lot of revenue at stake, explains especially as those sites proliferate in 5G Fluet, and a lot of it revolves around who networks. However, 5G changes what’s 5G specifications split the BBN into two handles the network slicing. “They could by possible because it standardizes on Ethernet further functions: the central unit (CU) and guaranteeing a better quality of service for very as a communications protocol between the distributed unit (DU). While a virtual advanced services that are demanding very these components, explains James Young, CU can control multiple base stations from special network performance. They can then director of data centers at CommScope. commodity equipment in the public cloud, sell those services at the higher price because Instead, carriers are moving to centralized, the DU must handle extremely low-latency they have higher value for their customers.” cloud-based, converged RANs (C-RANs). functions such as frequency switching which are crucial in a 5G environment. Telcos, hyperscale cloud providers, “With everything becoming Ethernet- and third-party data center operators are centric under 5G, now you can start to move “Most cloud and infrastructure standard all vying for position in this space, warns some of those base band pieces further stuff cannot support that level of processing,” Gabriel. “There’s so many different types of toward the center, and scale those up to Gabriel says. “So, then you need to put companies that see opportunity here and control a multiple number of sites from one accelerators into your cloud to offload some they’re all scrambling for position, and they instance.” This leaves only the radios at the of those very specialized requirements.” all have different cards to play.” cell sites and helps reduce costs because carriers can invest in fewer BBUs. One option is to use GPUs to accelerate She warns that data center providers those functions, and Ericsson is working who fail to grab part of that network slicing There’s another accompanying trend in with NVIDIA on this application. capability could be at risk. “Companies like BBU development: the virtual RAN (V-RAN). that could be squeezed into a commodity This virtualizes the BBU, enabling it to run Telcos investing in this transformation role,” she warns. on commodity hardware and creating hope to monetize it by supporting enterprise opportunities for carriers and 5G users. clients directly with these networks, possibly On the upside, the virtualization of the incorporating them as tenants in their own RAN puts everything up for grabs. Fluet Virtualizing the BBN offers potential edge data centers. They want to move sees an opportunity for enterprises running vendor independence. RANs have been beyond purveyors of communication private networks, either completely self- proprietary affairs with dedicated hardware, pipelines to grab a slice of lucrative services managed or with the help of other data but the Linux Foundation’s Open-RAN revenue. Gabriel isn’t sure they’re all up to it. center providers. They could even create (O-RAN) Alliance is defining standards for their own virtualized 5G cores, he explains. hosting virtualized C-RAN functions in the “The telcos have it in their heads that cloud. This would enable carriers to piece they’ve got the perfect set of locations for The opportunities for value-added revenue together solutions from different vendors. edge compute, because they’ve got all are huge, and the race has barely started. these central offices and cell phone sites,” Slicing Up the Network she says. “They do have that sort of physical Danny Bradbury is a technology journalist setup, but they don’t seem to have any with over 20 years’ experience writing The other opportunity is enhanced of the other real skills and relationships about security, software development, network functionality to support enterprise compared to the cloud web scalers.” and networking. applications. Bringing software-defined networking capabilities to the edge makes it possible to control traffic more effectively within the cellular network using a technique called network slicing, explains Fluet. Network slicing configures the network to support quality of service for specific applications like videoconferencing or industrial control at the edge. It all happens in software. “That’s much more efficient because previously, if you wanted to have a very stringent service level agreement you would typically just over-provision like crazy,” Fluet says. There are challenges in all this, explains Caroline Gabriel, research director and co- www.AFCOM.com 21

Getty Images BEST PRACTICES: JYOOUURRNEEMYECRHGEICNKGL5ISGT By Danny Bradbury 5 G is still an early development. assumptions and demonstrate quick wins. Telecommunications carriers commercial relationships underpinning 5G, Leave the mass drone delivery networks and are still in the early deployment and combine this with their understanding self-driving delivery trucks for later. phase, and the standards for of existing best practice in IT. Here is a some elements of emerging checklist of things to consider before Optimize for low latency: Data infrastructure are still barely embarking on your 5G journey. centers must be prepared to support dry on the page. With this in mind, many the low-latency applications that 5G and of the best practices for deployment are Be realistic about applications: “A lot edge computing applications will offer. still emerging. of people are very starry-eyed about the Companies might have micro data centers potential applications,” warns Caroline designed exclusively for low latency at Experts draw on what they know about Gabriel, research director and co-founder the edge, but this isn’t the only way to the emerging technical practices and at analyst company Rethink Research. Start go, explains Jacques Fluet, director of the small, with simple use cases to test your 22 www.AFCOM.com

Network Service Assurance Program at the IT. It marries software development and That’s a fairly new concept for many of the Telecommunications Industry Association deployment in a seamless process that relies data centers.” (TIA). “You may have a mix and match on the ability to configure infrastructure by of various architectures within your data programming it. Protect your value: There’s a lot of work center at the edge, or maybe a sub part of to do before data center operators get to the data center that is defined specifically “We talk about the CI/CD approach for this level, and they’ll need specialized radio for those special application.” network infrastructure,” he says. Companies networking support to make it happen, either must be ready to work with the service from a telco or a systems integrator, or both. Some data center providers are already providers managing the 5G infrastructure preparing for this. Equinix has adopted to support that. Enterprise customers must balance the segment routing in some of its data centers, need for those high-end skills with the for example. Prepare for dynamic orchestration: As value they’re prepared to give away, warns companies mature their 5G capabilities, Gabriel. She likens it to the early days of Design for lights-out operation: they should embrace the promise of the enterprise Wi-Fi when companies bought Automation will be key for companies self-configuring network that can adapt to in third-party consultants to manage the managing dozens or even hundreds of data support specific application needs, says radio element. centers across edge-based environments, Fluet. This depends heavily on the software- explains Ian Hood, chief technologist defined networking capabilities that they “The big companies will potentially take for global service providers at Red Hat. built before. an awful lot of the value out of the deal and “We must be able to drop services in not leave very much behind for the data and reconfigure them on the fly through “What it means is that there is going to be center,” she says. “My warning is not to give programmable capabilities,” he says. A a way for the network to auto detect what away too much of the value of the deal just lights-out operation at the edge is the goal to is needed, and reconfigure itself to provide because of the complications associated make these highly distributed environments the best possible services at any given point with wireless networking.” scalable and repeatable. in time,” he says, adding that AI algorithms will drive that activity in the future. “So, Danny Bradbury is a technology journalist Converge IT and networking functions: applications running on servers in the data with over 20 years’ experience writing That automation relies on consistent, centers might be modified, turned off, about security, software development, standard operations, which is what cloud- and turned on [automatically] in real time. and networking. native operations are designed for. 5G networking is built from the core outwards Getty Images with cloud-native computing in mind. All of their services run in the cloud, which is what gives their users access to software- defined networking capabilities. Hood advises organizations to take advantage of this, migrating legacy IT networks to cloud- based technologies and ensuring that they interoperate with virtualized networking at the edge. The same applies in industrial IoT environments. Companies can convert legacy control protocols to telemetry streams using virtualized gateways running on common computing platforms. Prepare for deployment pipelines: Done properly, this level of convergence will unlock the kind of software-defined networking capability that some companies have already seen at the core, extending it to the edge. Hood describes the continuous integration/continuous deployment (CI/CD) approach to rolling out applications that is becoming increasingly popular in enterprise www.AFCOM.com 23

RENEWING COLOCATION CONTRACTS: GETTING MORE AND SPENDING LESS By Kirk Killian M any data center managers are tasked with renewing process is very occasional - perhaps once every five years for data colocation contracts that are expiring soon, which can center managers - but a frequent task for colo providers, creating a be challenging. perceived experience and negotiation leverage imbalance working Successful colo renewal goals include reducing against the data center manager. costs, improving services, tightening SLAs, and creating contract flexibility. However, most data center managers are not However, in reality, enterprises can successfully reduce costs experienced in handling this type of contract negotiation. and improve services during most contract renewals if managers are aware of market dynamics and intelligently plan the process. Conventional thinking is that colo customers are at a distinct Managers can also materially improve the collaborative working disadvantage during renewal negotiations. Colo installations are relationship with colo providers during a contract renewal. This considered “sticky,” or difficult to relocate, since it can take over article discusses colo renewal strategies, improving contract terms, a year to methodically plan and move a data center. The renewal and getting internal credit for a job well done. 24 www.AFCOM.com

Understand the Market forward best positions users to reduce • Does the facility offer “direct connect” future costs. circuits to public cloud providers, private To successfully negotiate, it helps cloud services to assist in a gradual to understand the background of Users should also specify their required conversion to cloud, or public cloud market fundamentals. reliability metrics, cooling density for presence within their data center campus new hardware installations, and network or metro area, each smoothing the process Almost 300 mW of new high-reliability options to support ongoing operations. for future cloud adoption? colocation capacity was under construction in the US in early 2020, improving the Users should determine the PRICE ESTIMATION establishes supply-demand dynamic for users. certifications, audit/compliance processes, local market rates for similar colo space and security/managed services desired, and services. Many customers can get Many existing colo facilities have especially in an environment where many significant rate reductions for renewals by available space for re-lease, due to companies are shrinking IT headcounts to discussing lower prevailing market rates enterprise over-commitments in their initial control costs. with their existing colo provider, but only contracts years ago, coupled with move- if presented wisely. This is typically most outs of customers relocating workloads to One scoping task often overlooked is effective when the pricing discussion public cloud. Increasing cooling densities forecasting likely growth, which is difficult occurs well in advance of the existing have also created space availabilities, but essential. Managers should evaluate contract expiration, when there is still as users now place more hardware in their needs over both a short-term period ample time to threaten a relocation. less space, freeing up vacant space for of 2-4 years and a longer horizon of 5-7 new occupancy. years. Finally, budget constraints should Data center managers can obtain market be considered. pricing by initiating calls with sales reps for In most US and international locations, alternative colo facilities, describing 6-8 colocation pricing has decreased in the last FACILITY EVALUATION involves scoping parameters (physical size, critical two years. Many new colocation facilities measuring your existing colo facility and power, power growth, reliability needs, are being built much larger than they were provider against your future needs, typically telecom circuits, managed services, 5-10 years ago, and those scale efficiencies answering the following questions: unusual critical systems requirements, are being passed through to customers in etc.). Third-party advisors/brokers can both lower rents (as a function of lower • Does the facility provide sufficient also provide market pricing, conserving construction costs) and lower operating capacity (space and power) for current the data center manager’s time (and expenses (since providers can manage and long-term needs? anonymity, to avoid a barrage of sales larger facilities more efficiently than smaller calls) in the research process. ones). • Does the facility provide the reliability, cooling density, telecom providers, Many colo new customer proposals Finally, improved migration tools and network latencies desired for the are 20-30% lower than existing contract (software, services, and circuits) have foreseeable future? pricing for a similar quality facility in the lessened some risks and costs of relocating same metro area. If your existing colo suite to public cloud or other colo facilities, • Does the existing provider meet is in a high-cost location (e.g. San Jose, adding credibility to relocation options. minimum supplier requirements (facility However, most colocation users still prefer operations, financial stability, certifications, to avoid the costs and risks of moving an audit/compliance assistance, managed entire data center. services expertise)? Planning the Process The process for successful contract renewal negotiations involves scoping, facility evaluation, price estimation, and negotiation preparation. SCOPING is a key step in reaching your project goals. Correctly specifying the required space (how many cabinets), critical power, telecom networks, managed services, and cloud-migration tools is vital to planning a successful contract renewal. Contracting for more space and power than needed is the single largest source of colocation overspending, and accurately predicting that less capacity is needed going www.AFCOM.com 25

Getty Images Google, Microsoft and Amazon have all purchased land for data centers at the Agriport A7 businesspark in Medemblik, The Netherlands. Calif.), expand the list of colo facilities provider when combined with quantity power. Discuss with your provider that for price comparison to include regional flexibility going forward. overall market pricing has decreased since low-cost locations like Phoenix. the last contract was signed (assuming For most users, excepting those in you’re in a location - like most - with If your project exceeds a megawatt capacity-constrained locations or with decreasing rates). A twist on decreasing of critical power, you may wish to high growth forecasts, request only the unit costs includes densifying the complete a formal Request for Proposal quantity of space and power actually electrical/cooling capacity delivered to the process to a few providers including needed for the first two years of the existing premises, as it is usually cheaper the incumbent, sending a clear signal new contract term, coupled with an for the provider to deliver additional power that you are seriously evaluating expansion option to get more space without requiring additional space, subject relocation options. and power if needs grow over time. to cooling limitations. For example, you could reduce your During this process, instruct your data commitment by 200 kW and 30 cabinet To improve contract flexibility, center operations team not to discuss positions from your currently contracted enterprises should seek future contraction the planned renewal with the incumbent amounts (if those are surplus today), but options for both space and power. As provider’s staff. Sometimes your “contract get an expansion option allowing you to future needs for both are uncertain, lead” is making significant progress on add back those cabinet positions and especially in later contract years, ask successfully negotiating improved colo 200 kW if needed in the future. For many for the ability to partially reduce space renewal terms when a local IT staffer users, space needs are decreasing while and power commitments after the third shares leverage-killing feedback with server densities are increasing, so less year of a renewal. Even when users must the colo provider’s team such as “we physical space is needed than before pay a modest space reduction fee, cost would incur $5 million in unbudgeted to achieve the same overall computing savings can be substantial. This footprint move costs.” capacity. shrinkage model often dovetails with users’ plans to migrate workloads to NEGOTIATION PREPARATION is vital This “right sizing” of cage/suite public cloud. to successfully obtaining desired renewal specifications is typically the largest terms. The two most significant sources of source of renewal cost reductions. Don’t Start the negotiation process early, value improvement are space and power assume that the provider can’t take back allowing plenty of time for back-and-forth quantity reductions and rate reductions. space and reduce power, as they usually discussions. Patient negotiators do better! prefer retaining most of your business to Seek concessions multiple times; some Quantify the minimum and likely losing all of it. colo providers decline initial concession quantities of space and power actually requests, only giving serious consideration needed during the new contract term. Request lower unit pricing, which is after the customer negotiates in earnest. Many users can reduce overall costs by usually charged in $ per kW of critical giving back space and power to the colo 26 www.AFCOM.com

Additional Strategies to the provider. This shrinks colo spending Use the calendar intelligently since Improve Value and readies enterprise applications for some colo providers are eager to “make future migration to public cloud, all within their sales numbers” near the end of a If the colo provider won’t reduce the physical confines (and, therefore, quarter or fiscal year. Suggest an additional occupancy rates, seek several months “free audit/compliance control) of your colo “discount” if a renewal is signed by year rent” upon the new term commencement, premises. end. However, don’t promise a signature since some providers will grant incentives date and miss it, which creates ill will with to secure a long-term renewal. In other Spend-shift rights can also cover the provider. cases, providers are willing to increase the workloads moves to other facilities quantity of space or critical power delivered operated by that colo provider - such Engage an experienced advisor/ or provide more managed services (private as international - or to locations with broker to assist in the renewal process. cloud, professional services, free cross- lower utility or telecom rates, reduced Advisor/broker fees are paid by the connects, etc.) for the same total cost, latency, or better suitability for business colo provider (rather than the enterprise providing better overall value. continuity. Leading colocation providers customer), and those fees are typically have significantly improved their private far lower than the net contract savings If operating efficiencies have improved cloud and migration tools lately, assisting realized by the customer. Some data or electricity pricing has decreased, corporate customers evaluating public center managers realize that they are seek a reduction in PUE ratio or the cloud adoption. at a market knowledge and negotiating electricity pass-through rate, delivering experience disadvantage during colo line item savings. Enterprises can also improve service renewals, plus advisors help conserve a level agreement terms during contract busy manager’s time. Seek reductions in rates for blended renewals. Some customers rush SLA telecom circuits in renewals. At many reviews during their initial contract Inform your corporate legal several large colocation facilities, new telecom execution, and now wish to tighten trigger weeks in advance of the renewal process, carriers with low pricing have recently definitions, increase breach credits, so they can provide contract review when installed lit circuits into meet-me rooms, and improve cancellation options for needed. Inform IT procurement and and colo providers have lowered their downtime events. senior IT management if they will need aggregate costs for blended internet to sign off on renegotiated terms to avoid capacity. (Also, seek reductions in your Enterprises can also seek cost reductions unnecessary delays. third-party telecom circuit costs, leveraging before current contracts expire, usually the aggressiveness of upstart carriers to in conjunction with extending contract Not all renewal goals are financial, reduce pricing from incumbent carriers, or durations or adding capacity or new though, and a contract renewal is consider switching carriers.) managed services. also a great opportunity to improve the overall relationship with the colo Seek reductions in cross-connect At older facilities, review contract provider, which can deliver operational fees; colo providers have usually fully language and search for capital benefits. Managers are encouraged to amortized their costs for meet-me expenditure sharing requirements that explore ways to improve the provider room cross-connects during the initial may be triggered by future replacements relationship and modify business contract period. of major critical systems components practices or communications in a like UPS modules, chillers, cooling win-win manner. Request spend-shift rights to move towers, or CRACs. Renewals are an spending from colo occupancy charges to opportunity to remove any onerous Finally, document your planning process, other services or locations. For example, reimbursement provisions, which can quantify the savings to your organization, a customer could get pre-approval to arrive years later as unanticipated (and and note contract improvements achieved shift 50% of monthly charges into unbudgeted) costs. through your successful plan execution. managed services or private cloud from Sophisticated data center managers can get credit from IT management for If operating efficiencies have improved or electricity preparing and executing a successful pricing has decreased, seek a reduction in PUE ratio renewal plan. or the electricity pass-through rate, delivering line Kirk Killian is President of Partners item savings. National Mission Critical Facilities, advising corporate clients since 1999 in data center selection and procurement worldwide, and can be reached at [email protected]. www.AFCOM.com 27

HOW DO OUR DATA CENTER COSTS COMPARE WITH OTHER DELIVERY OPTIONS? By Tad Davies Determining what your data with other delivery options from a cost Decide what you will be measuring. center is actually costing perspective? This is largely a function of what provides your organization presents the most value and what answers significant challenges. However, • What metrics should our organization management seeks. You will want to understanding the true financial be tracking relative to costs? group costs by category and determine landscape enables accurate, fact- in advance how extensive an effort will based decision-making. Often costs • How might we structure our reporting be required. Most begin with the facility are compartmentalized among different to gain further insights into our costs? as a starting point. The categories in departments. Some costs are simply not Figure 1 on the next page are listed in recognized at all. • How can we use accurate information order of relative ease of creating a cost to make better short-term and model. For this discussion, we will focus We cannot make informed strategic long-term decisions? on facilities. decisions if we do not have the right data, or if the data we are using is wrong. Many This article will provide you with step-by- Determine timeframe. The useful life of a of us are asked the following questions step guidance and a practical approach to data center is typically 15 years. While a great and expected to provide accurate answers: creating a Data Center (DC) Cost Model. many of them are older, statistically what you Just keep in mind, you will still have to do will find is that after 15 years the physical • What are the all-in costs of operating the homework. So, here we go ... infrastructure needs to undergo multiple this data center? aspects of equipment replacement. As you Approach think about data center cost modeling, a • What are our projected data center longer view provides better perspective. costs in the future? First, here are some suggestions Therefore, document annual costs along regarding your approach. It’s important with at least 5- and 10-year perspectives. • How does our investment in data that you get it right from the start, or your center compare with our peers? final cost model may be inaccurate. That said, start here. • How does our data center compare 28 www.AFCOM.com

Leverage existing available talent. Figure 1 Information Uses Make things easy on yourself. Chances are Category Benchmark against: a) other data centers in your organization. because you are in IT, you don’t have much Facility b) other enterprise data centers. c) colocation offerings. finance experience. Maybe it’s limited to Benchmark against: a) other enterprise data centers. b) assembling your data center budget or People Managed Service Providers (MSPs). submitting line items for it (more about that IT infrastructure shortly). The absolute first thing you need Communications Same as above, plus SaaS providers to do is enlist someone from your finance Applications team to help you with this endeavor. That person not only knows a lot more than you, your model will be more granular than a determine total data center electrical costs. but he/she knows how best to package and budget, the accounting reference number So how do you get that? present the data! This person can: will appear on more than one line-item in the data center cost model. For the facility load, take the power • Recommend, or better yet, produce the reading from the ATS(s) (Automatic information in a format that stakeholders If the cost item is being calculated, Transfer Switch) serving the data center. inherently understand. Ever get a report provide the formula/methodology you Here, we assume that the ATS serves just that is difficult to read? The data is correct, used. The goal is to develop a transparent your data center. If no additional areas/ but it is not organized in a way that your model that can be improved as additional/ equipment are supported outside the organization consumes data. The initial better data sources become available. data center, you will have to factor them response is often to discount, ignore, or Additionally, include all possible relevant out (and remember to document your possibly reject it. Or maybe, at best, the line items, not just those for which you assumption.) The ATS may have a meter executives tell you to reformat it. have numbers. If the data does not exist, to access the data. If not, look on the enter it as a zero cost. That prompts the generator inspection reports to see the • Provide guidance in what matters to discussion of whether this category should reported load when the tests were done. your organization. As financial priorities or be included. In either case, it is best to have several interests change over the years, the finance data points mapped out monthly for an person is most likely attuned to the latest Leverage a cost modeling tool that’s entire year. This way, seasonal fluctuations thinking by management. easy to understand. Excel. Enough said. can be considered. For IT load, use the UPS load that corresponds with the load • Save you hours of work, especially if Working Through the Process data from the ATS or Generator Inspection they help with doing the homework! Report. Once you have the data, use UPS Let’s review and address some of the load x PUE x Utility Rate ($0.##/kWh) x For those organizations without much cost categories. 8,760 (hours in a year). bandwidth in their Finance Departments, this is a great exercise for an intern. General Cost Items. These are aspects Power usage effectiveness (PUE) is of costs that apply across the board. a ratio that describes how efficiently Leverage existing available data. a computer data center uses energy; Much of the data needed by your team Escalation. Since your model will specifically, how much energy is has and works regularly. A key source is encompass multiple years, you will need used by the computing equipment the budget for the data center. This will to decide how to apply cost escalation (in contrast to cooling and other provide you with costs currently being or inflation. The simplest method is to overhead). Anything that isn’t applied and how they are categorized provide an annual escalation value across considered a computing device in as well as tracking codes for each the board. The other extreme is to apply it a data center (i.e. lighting, cooling, category. However, some of it will come by line item. A happy medium is to apply etc.) falls into the category of facility from other sources, particularly if you escalation across the board and alter those energy consumption. PUE = Total are not in a freestanding data center. in which you have the most knowledge. Facility Energy/IT Equipment Energy. Sources include facilities (e.g. utilities) and finance groups (e.g. actual costs vs. Depreciation. When including Diesel fuel. You may have a third-party estimated costs from your budget) as well equipment replacement in the cost model, fuel distributor, or your generator service as third parties, such as your equipment you’ll also need to consider how your contractor may serve as one. They should maintenance providers. organization depreciates assets. know and be able to tell you: Whatever source or methodology Utilities. If you do not already have this • The typical consumption annually you use, document from where it was information, check with your facilities or based on testing (regularly scheduled generated! Use actual costs from finance/ finance departments. accounting. It helps if you use the same nomenclature for cost line items. A valuable Electrical. From an electrical reference is the accounting-assigned cost perspective, not having your data center codes. Most of the lines in your model metered separately will be challenging. If should have this reference; and since this is the case, consider this methodology: use PUE (Power usage effectiveness) to www.AFCOM.com 29

testing + minor and major PMs + “all-in” cost covering a myriad of building Assembling the Data load bank.) services including a rent allocation for your data center. Once collected, organize the data by • The consumption associated with subject matter. Consider adding comments unplanned run-time. You’ll want to review Organizing Costs by Facility- in the data table for particularly large a five-year period to determine an average Centric Category and numbers that appear irregularly, like annual cost. The year-over-year values Major Asset equipment replacement. could fluctuate significantly if you have experienced an extended outage in one Stakeholders are most familiar with Leveraging Data for year. Again, state your assumptions! organizing cost by facility-centric Business Use disciplines and/or major facility asset items. Facilities Staff. With freestanding data To get the required granularity, consider We all have recognized how formatting centers, accounting for facilities staff is using both in a hierarchical format. Invest data can help us visualize it better. That’s straightforward. In a shared real estate time setting up and reviewing the structure why you should consider producing various model, it can become more difficult. If among stakeholders to ensure that all charts that bring home and illustrate points your facilities department is charging you concur and, most importantly, understand to management that hopefully move them for work performed, then this is simply it. The team will be using this tool (Figure to act. (images are for examples only) an exercise in obtaining that data just as you would for a third-party contractor. 2) for a long time. Review, Improve, and Master At some organizations, facilities support might not be charged, but this remains a Asset Changes and End-of Life Once completed, review your tool with cost it bears to service the data center. management to ensure they understand it Under this scenario, request data on Are you are compiling the total data and they appreciate its value. Be prepared hours worked annually and average labor center costs over the next 10 years or so? to make changes to better suit their ability rates. If so, you should factor in major repairs to absorb the information. Invest effort and large ticket replacement items. Your to continually improve accuracy of data Building. Freestanding data centers vendors and contractors can provide that and data capture methodologies. Keep also have direct costs such as general information. You’ll also need to include management informed of improvements as building maintenance (e.g. lighting, HVAC, equipment that will reach end-of-life. Here it will keep them engaged and it will reinforce custodial services). Consider exterior costs are some rules of thumb to follow with the tool’s value (and yours!). Keep up with such as land care and snow removal. Other respect to the useful life span of equipment. new industry metrics, both data center directly related services, like security, may Keep in mind, however, that you will need centric and your particular industry centric, be included. Your organization may use to verify relative to your specific situation. so you can include them in your reporting. a per square foot metric that reflects an Master the data so you can have a seat at the table and influence the outcomes! Good luck! Figure 2 Tad Davies is president of Fodere Major Category Minor Categories Sub-categories Sub-sub-categories Consulting. He has been focused on Architectural Parts data center strategies since 1987 and Work & Maintenance, Access floor, consults regularly with data center and Replacement cleaning, executive teams. He advises clients on Parts/Equipment room integrity business centric strategy issues such as consolidation, colocation selection, build Mechanical Work & Maintenance, CRAC/CRAH, heat Parts, vs. buy, cost modeling and facility centric (including Replacement rejection, CFD compressors, fans strategy issues such as risk assessment, Plumbing) Parts/Equipment modeling, PM, new data center programming, owner’s Electrical major repairs Batteries, representation, and energy improvement. Work & Maintenance, UPS fans/capacitors Tad is a board member of AFCOM’s Fire Protection Replacement Switchgear, Data Center Institute. He graduated from Parts/Equipment generator, ATS, Pre-action, Tulane University’s School of Business Systems UPS, PDU/RPP, air sampling, with a Bachelor of Science degree Work & Maintenance, IR scanning, PM, gaseous agent in Management. Replacement major repairs Licensing, testing Parts/Equipment Work & Maintenance, Detection system, Replacement suppression Parts/Equipment system, PM Equpiment monitoring, BMS, access control, cameras, PM 30 www.AFCOM.com

UPCOMING LEADERS LAB WORKSHOPS AFCOM LEADERS LABS UPCOMING WORKSHOPS: This year we’re going digital! Our labs will be the The Role of Hyperscalers in same exclusive gatherings of thought leaders Your Data Center Strategy driving innovation, market awareness, and the Virtual Workshop advancement of data center technologies. October 14, 2020 AFCOM Leaders Labs are a series of workshops that bring together an intimate group of data center professionals who explore challenges and develop responses to the most critical issues impacting data centers today. Business Continuity and Disaster Recovery Best Practices for Maximum Data Center Uptime Virtual Workshop October 20,2020 Data Center Outfitting and Energy Planning for Optimal Building Performance Virtual Workshop October 22, 2020 The Data Center Balancing Act from the Core to the Edge Virtual Workshop November 12, 2020 For more information on how to participate or sponsor one of the labs, visit leaders.afcom.com.

We guide IT pros through understanding and managing next-generation networks, data centers, and cloud architectures. Join Our Community Today and Receive the 2020 IT Salary Report NEXT RESEARCH REPORT JANUARY 2020 | $499 IHTtooSwYaolaDurroyPSYeoueurrvsCe?yo:mpareIT SaTlarriehynt:ciaeHmcsopopwraeeeamiDlncaoaidfgYporsooiueeniwrtnCshnooIghmTsemcplayaaoprrytetebimtrotoiwohusYnposeiaul,srtayrtPttdroeorheeaosrnesrtntaeioortgkefwv,etcaahaohalenubrnitdercuohrspl.tUoiotienig.cSeetyahrs..lsesI2l,To0oeaop2rnnkgr0vdoaiarfmnuoetilnsztathisammyeitoibeaiornntenecat,lutlysthhr,heroaaiefwrtbnethietscesuytrraernet OLOR OISPUUABL CE. TABLE OF CONTENTS How do you compare to PREVIOUS NEXT your peers? Figure 7 APreeyeouraSwaarleawrhyat yCoourmmopstacormispaoranble peers earn? JANUARY 2020 Figure 8 38% 23% raeoOnOlfadvyteovioorutanehrlsle,jrhorhiabpao,s?wlipnlescSacluttasidsiofitnfegiydscoaofurmareepcymeotnupislowaotyinitmohnea,nlblteanspeefictsts 20% 2% 44% 39% 37% 22% 36% Total 41% 13% ObWveeaetahnmowr“rawaesnael4lotfhe,rn4ierdibtl%tDemssBeaao(oatsFao2ednfof:i:di/Is01nfgeorw2ftou%5wnufrhwmmiIorooITtaaTmto,esoimtoefntpnr7hWateepawhrene)dnrer.doekoyoe.6”2ifw1mfr06wee1oOmh9smeseeUfosnnn.sfrSoiee.oisIsTtsonrhanStaeanaieoldiacdldrastyoon?SlayumtsorhwaevWe,preswyaetorarhhfaaer8sea0ans,n0iydbdatIctTtelw?Poei3sr,omeff6eip3res%pse7Yeidoae%ennsarsrlesosadotidfITmrSfempuoeFoipermrrnsftmato(lf4tahooetabfcl6tfe(yeeru1/iheo%mrrs9rudnss(%m)6ee,eiooin9ond)mbrtn%rc(ueeFevarlcdnu)eein,lagsedlrctatyfueoiitsot(idrsoms2enean2Ne/hapts%t9Ominerasha)adf)n.i,iipelnhytohsd-aianptnph(gi(aFn1depi5sidrgy(%ut2asu1urw9)ams%irt%wnpeiiotacehi)nrt8,ecMht)ptat.hcr(stlhe8ehTeios1oioemr%npfticriebft)iocb,hcu(om3eea4rms9itn0r-ieo%e1pe-n(fikme)t,s)n--(3(4sb8mc0b%ah%iasne),te)da,tDetutapjihtorolaaes:ebnIynifr(to4h(roaom51efatp%8tmNiSsoi%Vnmne)aa,oWeuit)oliersot,caseyrnpfiksharbS2eyhlta0aaed1etl9nntl2aierosUd4eenfi.nS%fe.dgtikIdTth(ensS3beaol7a(moer4w%yfnS8lue)eja%r,ovfdebbiyt)gj,ooso/fVre.uf8beDel40et/0rsHai6csyxIprT%tseoeDi1Phaobrrm5itoesnevilfs%eesi’aspsfirassilieatwobuijdnsnoweiaofilliysbdtehrysdka:t RESEARCH REPORT Download the Report: bit.ly/salarydcw 11 Check Out Network Computing’s Sister Brands Inspiring. Informing. Educating. Check Out Network Computing’s Sister Brands Inspiring. Informing. Educating.