Data Center is our focus

We help to build, access and manage your datacenter and server rooms

Structure Cabling

We help structure your cabling, Fiber Optic, UTP, STP and Electrical.

Get ready to the #Cloud

Start your Hyper Converged Infrastructure.

Monitor your infrastructures

Monitor your hardware, software, network (ITOM), maintain your ITSM service .

Our Great People

Great team to support happy customers.

Friday, April 15, 2016

10 Ide Optimalisasi Biaya versi Gartner

Dalam kondisi ekonomi yang belum pulih, diperlukan strategi khusus untuk berhemat (bahasa kerennya "Optimalisasi Biaya"). Berikut Gartner memberikan pandangannya:


Top 10 IT Cost Optimization Ideas

Develop a plan before a crisis hits and budgets are suddenly cut.
It’s the year that your company’s customer self-service portal project is finally set to get underway. Then the news comes in from the CFO’s office: Everybody can expect midyear budget cuts. The portal project suddenly looks like a nice idea that will need to be moved to the back burner.
That’s the “wait-and-see” approach to cost optimization. Now consider the same scenario, under the leadership of a proactive CIO. Since the Great Recession, the CIO’s overall strategy has included conducting a cost-benefit analysis of all initiatives that require IT services. He already has a list of cost optimization initiatives that includes application rationalization. These initiatives would serve the business’s interests far better than delaying the portal project (which is projected to cut customer service costs while helping with customer revenue and retention goals).
“All organizations attempt to optimize IT costs,” said Jim McGittigan, research VP at Gartner. “But those that do it best focus on cost optimization as an ongoing discipline, not as a one-off exercise.”

Create a transparent view of costs

As a CIO, your first priority should be to make all IT-related activity costs transparent. Next, benchmark how your organization’s spending compares to your peers’. If, say, your data center costs are above industry averages, that area might be a potential candidate for cost optimization.
Above all, don’t approach cost optimization as a one-time activity. “Regularly scan the marketplace to stay abreast of what other organizations are achieving to gain knowledge of what is really possible,” advised Mr. McGittigan.

Consider “top 10” cost optimization ideas

In the customer portal example, the CIO had a running list of cost optimization ideas that would be of greater benefit to the business than delaying the project. Mr. McGittigan shared the Gartner “Top 10” list of cost optimization ideas:
  1. Create a shared-service organization for some or all IT services
  2. Centralize, consolidate, modernize, integrate and standardize technologies
  3. Leverage cloud services
  4. Increase IT financial transparency to better manage both supply and demand
  5. Utilize zero-based budgeting on the right cost categories
  6. Rationalize and standardize applications before cost-saving initiatives
  7. Optimize software licensing management and IT asset management capabilities
  8. Improve procurement and sourcing capabilities
  9. Invest in Mode 2 capabilities such as agile and DevOps
  10. Re-examine how end-user computing is delivered
Cost optimization initiatives offer a wide range of potential value that generally reflects the accompanying complexity and risk. For example, optimizing procurement processes is relatively straightforward, but offers low to moderate value. Launching the next big customer innovation or implementing cost-savings technologies with the business (such as launching a customer portal) will be more difficult, but generally offer more potential value.

Champion the ongoing initiative

CIOs are often reactive about cost optimization, said Mr. McGittigan. “Many times, they wait for cost reduction targets to be handed down before organizing cost optimization activities.”
Rather, CIOs should be the champion of this ongoing initiative. Part of their job is helping all parts of the business understand that while IT cost optimization includes cost-cutting, the focus should be on eliminating low-value activities. As a CIO, engage business leaders to ensure that optimization reflects business priorities and that you’re looking at IT costs in the context of end-to-end business processes.

Tuesday, April 12, 2016

Best Practices In Data Center Space Planning

The increased focus on efficiency within the IT world brings with it higher demands for thoughtful, well-planned spaces to house IT equipment with consideration given to future needs. As a result, focusing on current typologies can sometimes fall short of future requirements. Efficiency can be realized in a number of ways that affect planning and growth considerations. Most organizations try to predict their future growth needs but are not always able to see changes in the industry which may affect them down the road. Some of the most important factors to be considered in data center planning are:
  • Consistent footprints across platforms
  • Utilization/consolidation/virtualization of IT systems
  • Network, power, and cooling typologies and planning
  • Future considerations (growth or collapse implications)
This article examines these facets of data center space planning looking at various options available today in these areas and also positive and negative attributes to consider when looking at various options. It also examines impacts of these decisions on performance within a hypothetical space. Increased pressure to focus on efficiency can prompt changes in an effort to improve a space. Sometimes such decisions have adverse effects on operational and capital costs. But decisions can be made which not only improve efficiency but ultimately also provide a lower investment in capital improvements as well as operational costs.

Typical Equipment Footprint (TEF)

One of the biggest barriers to effective data center layout planning is having an understanding of the IT equipment that is proposed for the space. In most of today’s data centers a high percentage of equipment is rack-mounted servers. Most of the racks which hold this equipment are based on a standard width of approximately 19 in. for the support rails that hold the servers or other equipment in place. The rack enclosure then allows additional space for internal wiring, air circulation, and exterior panels and doors. Total rack enclosure widths end up between 24 in. to 30 in. (for larger spacing for wiring strategies). Rack depths are also based on a typical depth between the front and rear support rails of approximately 29 in. Once again, more room is given in the total enclosure for wire management, airflow, and the front and rear doors of the rack. The overall depth of a typical server rack can be anywhere from 36 in. to 48 in. depending on the make and use of the rack.
Although a large amount of equipment in most data centers is rack mounted, there are still a number of other equipment types in the space which are standalone devices. This equipment can vary depending on the system requirements, manufacturer, and product. Typical standalone equipment in today’s data centers are storage devices (for SAN/NAS type arrays), mainframe and tape, or virtual tape systems for long term back-ups.
Another major IT equipment type typically found in today’s data centers are network systems equipment and distribution racks. Although some data centers place the network core systems outside the data center proper, often times these core areas (including primary and edge switching, distribution, appliances, and carrier equipment) are located directly and centrally in the data center. Most of this equipment is typically rack mounted, although in many cases these racks are open racks without any type of enclosure. These racks are usually single pole mounted (with only a single vertical support on either side of the rack) at 19 in. in width.
The different types of equipment present means that some variation in the footprint layout of IT equipment is common. Both storage and mainframes are typically wider than a 24- to 30-in. size and can even reach depths beyond 48 in. Typically, special planning of these systems must be considered as part of the IT layout,  not only to accommodate their size but also their airflow patterns, which can vary from the front to back airflow typically planned for. While this variation does occur, such equipment usually represents a smaller percentage of the overall equipment in the space. The most common planning footprint for an equipment rack is normally considered to be 24 in. in width and 48 in. in depth, or 8.5 sq ft (factoring up slightly for larger IT equipment). In most planning scenarios these dimensions will net the highest yield of typical layout space and allow for the most flexibility when planning out the data center space. This size also works well with an access floor system, which is typically set to a 24- by 24-in. grid (or in metric at 600mm x 600mm).

TEF Densities

When planning a data center it is important to understand the required power density currently being experienced and those anticipated in the future. Power density is a good metric to review both mechanical and electrical distribution solutions. Power densities have increased over time as IT equipment shrinks in footprint. However, these densities have not increased to the extent once imagined due to increased efficiencies in chip technologies. The best approach to planning a data center layout is to categorize the IT equipment into these three power density levels (more than one level may be present at the same time):
  • Low density. This level is more common in older spaces, smaller commercial sites, or sites with restrictive abilities to reach higher densities. This level would typically include TEF ratings of a range of 1 kW to less than 6 kW.
  • Medium density. This density is very common in today’s enterprise, government, and larger commercial sites. Often this is a result of an IT group’s conscious decision to avoid high densities due to the changes required to support it (spreading the load rather than stacking the load). This level would utilize a TEF rating range of 6 kW to less than 15kW.
  • High density. This density is less prevalent and often not maintained throughout a data center. Locations that do experience these densities are typically reserved for university or medical research facilities using HPC (high-performance computing), cloud service providers, and wholesale colocation providers. The rating range at this level is over 15 kW per TEF.

The levels noted above are often found mixed in the same facility. There may be a small amount of high density servers in a data center while an overwhelming amount of the total are low or medium density for the rest of the space. Most power and cooling solutions will work with both low and medium densities, but more consideration must be made when dealing with high densities.
It should be noted that the densities or kW/TEF should be based on actual power consumption, not nameplate power rating for IT equipment. Manufacturers of IT equipment provide information based on the maximum power consumption possible for the device. This occurrence is almost never realized in typical operation and also typically represents a fully configured, loaded, and utilized device.
In a high percentage of instances the connected load of a piece of equipment or equipment rack is substantially below the rated power draw of the manufacturer’s published information. The differential between the actual equipment power drawn vs. the manufacturer’s published information is known as the IT equipment diversity factor. In the past, the equipment diversity factor was sometimes as high as 60% (in other words the actual draw of typical equipment was only 40% of the manufacturer’s published data for the equipment).
Today’s IT equipment is more efficient through consolidation of equipment and virtualization of operating systems (locating multiple operating systems and associated applications on a single physical server which increases overall utilization of the equipment). This has driven down the diversity factor to the 30% to 40% range. It is important to understand a client’s current power loading (uninterruptible power supply [UPS] power loading vs. IT equipment nameplate ratings) to calculate the current diversity factor. This will allow for power growth planning that is not overstated and can be matched more closely to the actual power needs. This in turn better allows for the capacity needs of the mechanical system and distribution methodologies.

Typical Hot/Cold Aisle Arrangement In A Data Center

With the background information provided previously we can now examine typical techniques used in data center layout. In order to maintain an efficient plan to provide separation between supply air to the IT equipment and return air back to the mechanical systems, it is recommended that IT equipment be laid out in a hot aisle/cold aisle arrangement. This allows for supply air to be provided (typically through the access floor) to the front of the equipment where intakes are typically located in the cold aisle, and provide exhaust air from the back of the IT equipment to flow back to the HVAC system’s return in an open return fashion. At low and even medium TEF densities this typology can be effective, but this can be hampered by aspects of the IT equipment and how well the equipment rows are managed. There are a number of factors that can impact this basic technique and adversely degrade supply air temperatures to the equipment:
  • Equipment row gaps/missing blanking panels in racks: Hot and cold aisles are most effective when there is consistent separation between the airstreams. Any opportunities for mixing of colder supply air and warmer return air will dilute the supply air to the racks. Gaps in rows and a lack of blanking panels in the racks themselves allow for the mixing of air to occur.
  • IT equipment airflow deviations from a front to back direction: Some IT equipment does not operate with a front supply air intake and rear return air exhaust. Many storage arrays operate with a front intake and top exhaust path. Network switches often operate with a side intake and side exhaust. In either of these cases, supply air mixing with return air exhaust is much more likely to occur.
  • Location of HVAC equipment returns: The placement of HVAC equipment within the data center (typically CRAH or CRAC units) is important to help direct the return airflow. Should the airflow for the return be located in a manner that requires it to go past or through the cold aisle, then an opportunity for air mixing is introduced.
With these factors understood, basic layout techniques can be used to achieve reasonable results by maintaining hot/cold aisle arrangements in the data center, although certain aspects may not be able to be overcome. In cases where basic needs to provide good layouts are difficult (such as equipment types which are not conducive to front to back exhaust alone), other techniques can by employed to better achieve good airflow management and separation:
Hot aisle/cold aisle containment. In this strategy either the hot aisle or cold aisles are encapsulated to prevent these airstreams from escaping or mixing with the opposite type of airflow. Containment is typically achieved through a panelized system using a framework and plastic infill panels or a heavy vinyl sheet system which is hung from a horizontal bracket. These systems typically run from the underside of a dropped ceiling to the top of the IT equipment racks. Where racks are not present these systems may run all the way to the access floor.
  • Advantages: Limits potential for airflow mixing; Allows controlled and contained airflow for contained system; Mechanical units can be located outside data center (with plenum or ducted return).
  • Disadvantages: Rack uniformity or in-fill panels required; Doors at ends of containment required; sprinkler system patterning can be affected (fusible links may be required by AHJ); and affects working environment in containment area (colder or warmer temperatures experienced).

Return air plenum and IT rack chimneys. This is a simpler system than the containment systems described previously and exhibits many of the same benefits while eliminating some of the drawbacks. This system utilizes a return air plenum to which chimneys are run from the top of the IT equipment racks to the ceiling plenum. The HVAC unit return is also run to the plenum to draw the exhaust air back to the units. Chimneys are run from every cabinet.
  • Advantages: Limits potential for airflow mixing; allows controlled flow of return air back to HVAC units; eliminates the need for aisle doors; racks can be of varied heights; mechanical units can be located outside the data center.
  • Disadvantages: Equipment must be in an enclosure; sprinkler and lighting patterns can be affected; rear cabinet doors must be solid and increase exhaust temperatures within the rack.

Close coupled cooling systems. One of the easiest ways to assure lower potential for air mixing is to bring the supply air system to the IT equipment rack itself. There are a number of systems that can be located directly adjacent to the IT equipment rack (either over or next to) or capture the exhaust air as it leaves the rack (in a rear-door heat exchanger). These options offer the advantage of minimizing the distance from either the intake or exhaust to the cooling system and virtually eliminate the opportunity for mixing. However, they do have some issues relating to their use.
  • Advantages: Limits potential for airflow mixing; provides immediate cooling or heat dissipation at the IT equipment; reduces or eliminates needs for underfloor plenum.
  • Disadvantages: In-row and on-rack deployment affect rack densities (number of IT rack spaces per row) and depth (rear door exchangers affect rack depths); these systems typically require a heat exchange unit or distribution unit on the data center floor; there is typically no humidity control in these smaller units; in larger data centers, redundancy/reliability needs may drive up overall costs for implementation and lower TCO.

Air-handling systems with airside economization. This covers a number of different types of systems from more traditional building air handlers with economizers to enthalpy wheel systems and indirect evaporative cooling systems. In all these systems there are some common elements. They will reside outside the data center with a supply and/or return plenum directly adjacent to the data center (over or next to the space). They are generally pre-packaged systems with coordinated controls that are shipped and placed on-site rather than built on-site. Their efficiency is affected by outdoor environmental conditions and need to be matched to the climactic tendencies of the site.
  • Advantages: Can limit potential for airflow mixing with proper layout planning; does not require HVAC equipment in the data center; reduces or eliminates needs for underfloor plenum; in proper climactic conditions these systems can offer the lowest TCO.
  • Disadvantages: These packaged systems have large footprints that require exterior placement and access; these systems tend to have higher initial costs; certain types can have high water consumption needs; redundancy may be achieved through use of direct expansion (compressorized) operation.

Hybrid solutions. The various layouts noted above should not be considered exclusive to one another for the needs of data centers which may have mixed needs or fluctuating densities. Any and all of the layouts noted above may be used in conjunction with one another (some being more suited to work with others), and consideration of any of these should be made during the planning stages of any major data center renovation work or new construction planning.

Network, Power, And Cooling Typologies, And Their Effect On Layout Planning

A commonality with the layout techniques described previously is the planning for airflow to and from the IT equipment racks. Without allowing for this, data centers suffer from constant issues with general overheating and hot spots. There are considerations which must be made during planning for a data center relating to the network, power distribution, and cooling solutions which need to be explored in order to achieve the best possible outcome and proper operations.
Network/communications. Network and other communications wiring and equipment can greatly affect the data center layout and ultimate airflow to and from the IT equipment. There are a number of factors to review to make sure proper planning is achieved:
  • Network core: Location of the network core in the data center will greatly increase the amount of wiring required to come to and leave the space (use of copper vs. fiber can affect cabling size and amount). Consideration should be made to locate the core in a dedicated space outside the data center proper to reduce cabling loads.
  • Network cabling location: As data centers have evolved, network cabling has gone from a higher percentage of underfloor cabling paths to overhead cabling instead. By running the network cabling overhead, this reduces congestion in the underfloor air plenum (if access floor is used for this purpose), and also forces better cabling management with the cabling exposed to view. It’s important when planning for overhead network cable trays to coordinate with the IT equipment layout, as these trays should be run over the top of the racks. Tray height, cabling connectivity to the racks, and impact on other systems (such as lighting and fire suppression) should be considered.
  • Distribution switching: Network systems typologies can affect the pathways and number of cables required to allow for proper communications across the system. The use of end of row switching in lieu of top of rack switching can cut down considerably on the interconnectivity cabling requirements of the system. Also the use of fiber distribution instead of copper cabling can cut down on the cabling diameters and fill in cable trays, making the systems easier to handle and lighter overall.
  • Systems communications: In addition to the network needs of a data center, oftentimes IT planning fails to account for the various systems communications required for BAS/EPMS, security, fire alarm and detection, and other support systems for the data center. Often network security or jurisdictional restrictions require separate systems and/or pathways for this type of communications. Upfront discussions for these considerations are strongly advised.

Power distribution. Much of the discussions throughout the data center around cooling needs for IT equipment. Power distribution is equally important, but this too can affect airflow management. In laying out the power distribution in the data center the following should be reviewed:
  • Power cabling location: Traditionally, power cabling has been run in flexible conduit below the access floor to each IT equipment rack. In redundant power schemes, two or more cables can be run to each rack. The use of end-of-row remote power panels (RPPs) can cut down on cabling runs and distances. They also reduce underfloor cluttering and increase airflow.
  • Overhead busway distribution: As these systems are becoming more prevalent in the data center market, the use of busway distribution cuts down on the use of cabling and allows for easier change-out requirements for connectors when rack changes occur. Redundancy can be achieved in separate busways and the overhead use (in conjunction with overhead network cabling) frees up the underfloor plenum for airflow (if an access floor is even required).
  • Power distribution voltage and type: IT equipment manufacturers are making it easier to allow for 400V distribution to be used in the data center. If the equipment can accept this voltage, the need to transform power down to lower voltage is relieved eliminating equipment required (typically a power distribution unit [PDU]) in or around the data center, allowing more space available for the IT equipment. The use of DC voltages in lieu of AC power typically used is also being explored. Although both of these types of distribution are not widely used today, larger data center owners and operators are looking at these systems as a means to cut down on costs and increase reliability overall.


Cooling distribution: Although much of the concentration of this paper discusses data center layouts relating to mechanical systems, little has been discussed about the actual distribution methods including plenum make-up, perforated tiles, and other distribution means. The following are noted to be considered:
  • Access floor plenum: The access floor plenum as a supply pathway for cold air has worked in data centers since their inception. Over time we have seen the depths of these plenums increase. Largely this has been due to the congestion often created with other systems (network, power, fire detection and suppression, etc.) being located in the same area. Through extensive CFD modeling of underfloor cavities we have found that a clear depth of 18 in. is all that is required for good air circulation and static pressure requirements. This assumes the cavity is able to hold static pressure and only using perforated tiles for exit points for airflow. Access floor panel conditions and concrete slab sealing are important to maintaining an active plenum. Plenum depth, depressing the concrete slab, and other potential implications surrounding underfloor plenums should be explored.
  • Perforated tiles: Perforated tiles have long been used as a means to provide airflow directed at IT equipment racks. The original standard tiles only provided for 25% free area for airflow (in other words the tiles were 75% solid). These panels offered less rolling capacity than standard tiles and only provided airflow directly through the tiles (limiting airflow could be achieved through the use of dampers which impeded airflow even more). Newer perforated tiles allow for 56% and 68% free area of the tile allowing increased airflow. They also offer directing fins which turn the airflow toward the equipment rack rather than straight up into the aisle making better use of the air coming out of the floor. These tiles are made to meet or exceed the general floor rolling loads improving equipment use in the colds aisles. Airflow should be reviewed prior to planning being completed for data center layouts.
  • Cold aisle widths: Standard data center layout pitch (the distance from the center of one cold aisle to the center of the next as defined by ASHRAE) is seven tiles or 14 ft (4,200 mm for metric tiles). This allows for a 4 ft or two-tile cold aisle width. Depending on equipment densities, this aisle width may need to be expanded to three tiles (or 6 ft) if high densities are required. This also would affect item  “b” above in the perforated tile layout to assure proper coverage of supply air to the IT equipment racks.

Future Planning

This article focuses on many of the current data planning strategies being employed today. Although there are more possibilities, the ideas presented here represent the current mainstream thinking for data centers (both renovations and new planning). In terms of emerging trends, there are a number of planning techniques being explored and implemented that can be considered on the cutting edge of data center design. These ideas may become more mainstream as TCOs come down and more widespread acceptance takes hold:
  • Containerized data centers. Although many systems from different providers have been available for more than 10 years, these “data centers in a box” have not garnered the market attention once thought likely. As IT equipment densities and utilization increase, these may become more popular with quick time to market capability. Also, colocation providers are eroding the need for smaller commercial data centers to exist, lessening the attractiveness of owning your own box when someone else can manage it for you.
  • Elimination of access floor plenums. With the advent of cooling systems which can operate in the data center space (such as in-row cooling and air-handler/economizer systems), the need to provide an underfloor supply air plenum has been reduced. This, in combination with overhead network and power distribution, eliminates the need to use an access floor system altogether. This enables cost savings for both the floor itself and also in structural costs to depress a floor slab in new facilities or install stairs and ramps in existing sites. This situation has taken hold in many smaller sites and is becoming the norm in larger deployments for wholesale colocation providers and internet service providers.
  • Modular system deployment. Modular electrical and mechanical systems are similar to item containerized data centers above, but offer the flexibility to be used in both new deployments and existing sites. They provide improved development and construction as they are factory built to specific needs of the client. These are typically skid mounted for ease of transport, delivery, and installation. These plants can be designed and built for expansion over time and offer quick turn-around times from approval to delivery over traditional site built options. Space planning is made easier by having shop drawings to work from to confirm clearances and in cases where existing sites are being used; oftentimes the fabrication process can accommodate site specific needs if required. 

References

This article draws on information previously published in a variety of sources, most notably:
  • Schneider Electric — A Scalable, Reconfigurable, and Efficient Data Center Power Distribution Architecture (White Paper 129 Rev 1, 2011)
  • U.S. Dept. of Energy — Best Practices Guide for Energy-Efficient Data Center Design (2011)
  • ASHRAE — Thermal Guidelines for Data Center Processing Environments (2012, 3rd edition)
  • ASHRAE — Data Center Networking Equipment — Issues and Best Practices (2013)
  • TIA 942-A — Telecommunications Infrastructure Standard for Data Centers (March, 2014)
  • Internap — Critical Design Elements for High-Power Density Data Centers Presentation (Retrieved April, 2015)

Data Center Cooling Metrics

Modern data centers continue to evolve at a rapid pace. While methodologies and techniques for cooling continue to advance, some of the basic lessons that have proven themselves over time continue to be underutilized. New technology and techniques can often be helpful, but without employing fundamental airflow management metrics the full benefits of advanced cooling methods cannot be realized. Fundamental data center metrics have been the basis of many publications and presentations since the industry’s founding, but emphasis on the fundamentals has dropped off over the last several years because advances in cooling methods such as containment, free cooling, and evaporative cooling have held the spot light. However, applying the fundamentals is crucial to getting the best results, regardless of having a legacy cooling configuration or the latest advanced free cooling methodology. Therefore, experts in the field have placed a renewed focus on fundamentals and have recently broached the topic at many of the industry’s biggest events.
Airflow management (AFM), in a nutshell, is about improving data center airflow so the least amount of conditioned air at the highest supply temperature can be used to effectively cool IT equipment. The following metrics discussed can help you identify which fundamental items can be improved, thus increasing your data center’s cooling capacity, IT performance, and energy savings regardless of the configuration of your room or the cooling methodology being used.

 

Power Usage Effectiveness (PUE)

Created by The Green Grid, PUE has become the most widely used metric for assessing the energy efficiency of a data center. In fact, PUE data reveals that cooling infrastructure is the single largest consumer of data center power (typically around half), and therefore, the largest contributor to a high PUE value. Considered the highest level metric to look at overall efficiency, measuring PUE is a great starting point to measure data center performance and track changes/improvements made to a data center over time. Represented by the formula:

PUE =  Total Facility Energy 
           IT Equipment Energy

PUE is determined by dividing the amount of power entering a data center (total facility power) by the power used by the computer equipment.
While an extremely important tool, PUE cannot tell you specifically what to improve to make a data center more energy efficient. Additionally, PUE is not a standalone reference point that provides useful information when calculated infrequently. While the average PUE of data centers surveyed has been dropping over recent years, there is still a great deal of room for improvement. Additionally, there has been a growing trend of misuse of PUE. Many sites are calculating a partial PUE (pPUE) by not including all loads in the total site power but reporting it as total site PUE. pPUE can be a valuable measurement but should be reported appropriately.

Cooling Capacity Factor (CCF)

Cooling equipment consumes the most power in a data center behind the IT equipment. Developed by Upsite Technologies, CCF is a metric used to estimate the utilization of the computer room cooling capacity. By determining how well the cooling infrastructure is being utilized, you can identify potential gains as a result of AFM improvements and controls adjustments. This is fundamental to improving the cooling of the entire data center (free cooling, chiller plants, etc.) and has the greatest leverage toward improving your PUE. CCF is calculated by dividing the total rated cooling capacity (kW) by 110% of the IT critical load (kW):
Total rated cooling capacity is the sum of the running cooling units’ rated capacities. If all cooling units are running, then this will be the same value as the total installed rated cooling capacity. A CCF of around 1.2 is most desirable, although a score of 1.5 to 3.0 is most common. In the latter case, there is likely significant stranded cooling capacity that can be recovered through improvements to AFM.

 

Cooling Effectiveness

While many data centers have monitoring in place via a multitude of sensors placed in a variety of locations, very few regularly check to identify the effectiveness of the cooling for every U space of every cabinet in the computer room. This is important because hot spots can occur in very isolated locations that sensors can often miss. To help avoid this, infrared cameras and infrared thermometers should be utilized regularly to identify hot spots. It is also important to identify the percentage of cabinets with cold spots and the percentage of cabinets with hot spots so that you can determine which areas need focus.
There is a direct correlation in data centers between the range in intake air temperatures and the efficiency of the cooling infrastructure. Ideally, the difference between the warmest intake temp and the coldest intake temp should be 5 degrees or less. If not, there is room for improvement. ASHRAE’s recommended range for intake temperatures is between 64°F and 80.6°F. While intake temps below 64°F are not going to impact IT reliability, they are an indication that an excessive amount of energy is being used to cool the room. Hot spots (intake temps above the desired maximum for the site), are an indication that the cooling system is not effective and the IT equipment reliability may be compromised — a situation which needs to be remedied as soon as possible.

 

Raised Floor Bypass Open Area

For data centers that use raised floors, this is one of the simplest and most important metrics. It’s merely what percentage of the holes in the raised floor are in a “good” location and what percentage of the holes in the raised floor are in a  “bad” location. Good means that the air coming out of a tile is directly used by IT equipment. Bad means supply air coming out of the opening is not being consumed by IT equipment. The only good type of open area is the supply tiles (perforated tiles or grates) directly in front of IT equipment. The two types of bad open areas are unsealed cable openings under cabinets and around the perimeter of the room, and misplaced supply tiles (in open areas or hot aisles).
For example, if a computer room had one cabinet with one standard perforated tile in front of it with 25% open area (1 sq ft) and there was one unsealed 12- x 12- in. (1 sq ft) cable cut out, then the total raised floor open area would be 2 sq  ft. The raised floor bypass open area would be 1 sq ft or 50% bypass open area.
Although many data centers have made an effort to seal cable openings and other potentially harmful holes in the raised floor, very few have completed the job. These remaining openings can easily release significant flow rates of conditioned air which limits the capacity and efficiency of the cooling infrastructure. The goal is to have no bypass open area; the only openings in the floor being the supply tiles in front of IT equipment. It is particularly important to seal or (depending on the design) at least reduce the open area under electrical equipment, such as power distribution units (PDU) or remote power panels (RPP).

 

Perforated Tile Placement

The use of perforated tiles is one of the simplest and easiest ways to manage airflow in a computer room. However, few data centers do this well, despite the fact that perforated tiles can be a quick and relatively inexpensive fix to improve cooling. Even in well-managed sites, there is often still room for improving perforated tile placement.
There seems to be a growing trend of not adjusting the placement of perforated tiles to the actual load of the computer room. In a research study I conducted of 45 data centers across the world, only six sites (13%) had properly placed every perforated tile. This is especially sobering when you consider the amount of wasted energy needed to keep these data centers properly cooled.
The definition of a properly placed perforated tile is within two tile positions of IT equipment intakes. Conversely, an improperly placed perforated tile is typically any tile in a hot aisle or open area of the room. However, there are important exceptions. For example, if IT equipment has been mounted backward (from a hot aisle/cold aisle perspective) with the intake in the hot aisle, then a perforated tile is likely needed in the hot aisle until the equipment can be turned around. This should never occur in the first place but still does surprisingly often.

 

Bypass Airflow (Ratio Of Supply Airflow To It Airflow)

The definition of bypass airflow is any conditioned air that does not pass through IT equipment before returning to cooling units. The only way to improve bypass airflow is to reduce the total flow rate of air moving through the room via the cooling units. In many cases, the total flow rate of air supplied by cooling units is two to three times the total airflow rate required by the IT equipment. This much excess bypass airflow is often necessary to overcome poor AFM. However, if you improve the AFM, it may be possible to reduce the total volume of air flowing through the room. To identify how much bypass airflow is occurring in the room it is necessary to determine the total flow rate of air moving through the IT equipment and compare this to the total flow rate moving through all the cooling units.
Historically, blade servers have produced higher Delta Ts (∆Ts) than traditional rack-mount servers (“pizza box” servers). In other words, the cool supply air entering a blade chassis would exit as hotter air than would the supply air entering a pizza box server. This difference is described by the equation of heat transfer:

q = Cp x W x ∆T
q = amount of heat transferred
Cp = specific heat of air
W = mass flow
∆T = temperature rise of air across the heat source

When we normalize the terms for units we typically deal with, this relationship is described as:       

CFM = 3.16 x Watts
∆T

CFM = cubic feet per minute of airflow through the server
3.16 = factor for density of air at sea level in relation to °F
∆T = temperature rise of air passing through the server in °F

Based on this relationship, a 5kW blade server chassis with 16 servers and a 35°F ∆T would draw 451.4 CFM:
451.4 CFM = 3.16 x 5,000
35° F

In contrast, ten 500W pizza box servers with a 20°F ∆T would draw 790 CFM:
790 CFM = 3.16 x 5,000
20° F
In a data center with 1,600 blades (100 chassis), the servers would consume 45,140 CFM of chilled air (100 chassis x 451.4 CFM per chassis = 45,140 CFM) as opposed to a data center with 1,000 pizza box servers, which would consume 79,000 CFM of chilled air (1,000 servers x 79 CFM per server = 79,000 CFM).
Table 2 shows the CFM required to cool a kW of IT load relative to the IT equipment ∆T.
By estimating the average ∆T of the IT equipment in the room you can estimate the average CFM required to cool a kW of IT load. Then you can calculate the total IT equipment cooling flow rate with the following equation:
UPS load (kW) x average CFM/kW = Total IT CFM

The bypass airflow rate can be simply determined by subtracting the total IT equipment cooling flow rate from the total cooling unit flow rate. The total cooling unit flow rate can easily be determined from cooling unit specifications.
For example:
  • There are eight cooling units that each deliver 12,000 CFM
  • Total cooling unit flow rate is 96,000 CFM (8 x 12,000 CFM = 96,000 CFM)
  • The average IT equipment ∆T is 25°F
  • IT required cooling flow rate is 126 CFM/kW
  • UPS load is 325 kW
  • Therefore the total IT equipment cooling flow rate is 40,950 CFM (325 kW x 126 CFM/kW = 40,950 CFM)
Bypass airflow rate is 55,050 CFM or 57% (cooling flow rate 96,000 CFM – IT flow rate 40,950 CFM = bypass flow rate 55,050 CFM) (55,050 CFM / 96,000 CFM = 0.57 = 57%)

 

Conclusion

While these metrics may seem remedial to the experienced data center operator, they very often reveal opportunities for further improvement that have been overlooked in pursuit of the newest trends and technology. Starting with fundamental steps, like improving the AFM in the room by simply properly placing every perforated tile, can improve conditions to the point where significant energy savings can be realized without the need to invest in new equipment. Cooling unit fan speeds can be reduced and supply air temperatures increased to levels previously thought impossible, all without impacting the IT equipment intake temperatures. By employing these basic metrics at the outset of an efficiency evaluation or AFM upgrade, data center operators can begin to make accurate and necessary changes to their sites and improve the total cooling capacity of the room, often improving the reliability of the equipment and saving energy costs.

The Value Of Data Center Monitoring

If you are considering adding network monitoring to your IT system, or currently have a system in place, you may be asked to define its value from time to time. When looking at determining the value of network monitoring for enterprise networks and data centers, one needs to consider several factors with upfront and ongoing cost of the software and its value over time, being two major concerns. The natural first response may be “Let’s look at the data.” However, there are many soft factors to consider, and these are more subjective and difficult to quantify. So, while data is helpful, there’s no simple way to quantify the return on investment (ROI) for network monitoring. Take things a step further and look at other benefits to give you a broader, more complete picture.
Determining the value and ROI of network monitoring is a challenge, but there is tremendous value in going through the exercise. Naturally, the “hard costs” are easier to identify than the soft ones. These can include things like licensing and hardware costs, implementation costs, upgrades and module costs, and service and maintenance fees. There are also the more incremental costs of additional modules and add-ons, and potentially, the need to purchase new software to monitor new technologies.
Finally, you have to remember the costs of salary and benefits for the IT staff. How does network monitoring help with these costs? A network monitoring tool can help you to quickly and easily determine which versions of software and hardware that you have so that you can keep everything updated with the latest version, which helps overall efficiency. It can also help you to assess in advance what add-on modules you might need (or not need) helping you to spend more carefully, rather than buying addition modules too quickly that might end up not being needed after all.  Additionally, monitoring is a watchful set of eyes that works 24/7, helping IT to eliminate things like split shifts and “on-call” hours.

 

Painful Numbers: The Cost Of Downtime

A more significant — and painful — factor can be the cost of network downtime and failures. You may ask yourself “What damage (and costs) occur if a mail server or an e-commerce site crashes?” or “How much does two hours of network downtime cost our company?” It is virtually impossible to give an exact number — an outage at an online retailer during the holiday seasons is substantially more damaging than an outage on a Friday afternoon in August. But while exactitudes are illusive, industry experts have made estimates that convey the harsh truth — downtime can have staggering costs.
A number of firms have done studies on the impact of downtime. For example in 2014, Gartner estimated the average costs of network downtime to be $5,600 per minute, which means approximately $300,000 per hour. A study by Avaya estimates the range for the cost of downtime to be between $140,000 and $540,000 per hour. The takeaway here is that the cost of downtime can be significant. For some, that kind of data would be enough to convince them of the importance of network monitoring and its role in preventing downtime and failures. But there are more things to look at to determine the true value of network monitoring.

 

Don’t Forget About Soft Costs

Some other “soft” factors that are more difficult (or even impossible) to quantify include things like customer support being unavailable due to a network outage. How many customers and how much revenue is lost when online chat support or email support is not available, or the VoIP-based phone system that the customer service department uses is not working? When a customer can’t reach customer support, do they complain about the company on social media? Do they take their business to a competitor, never to return again? These things are hard to estimate, but the potential negative impact can have lasting effects.
Another example is if a website is unavailable. Depending on the website and how it is used, it can have different impacts and costs. For a web-based or online-only company, it may be more concrete and measurable — that may be there one (and only) channel for generating revenue. For a branding or image-focused website, there is lost marketing value or brand damage that could result from an outage.
These are external factors, but there are also soft costs internally that can have a negative effect on employee productivity and morale. An early-morning email server outage on a Friday may cost an entire workday, while poor network performance during the workday creates additional frustration for employees, lowering morale and, in some cases, making them look for new opportunities. It may seem rash, but if an employee can’t get their job done because the network is freezing and crashing, they are not going to be happy.

Long-Term Strategic Benefits: Reduced Workload, Time Savings

Beyond the costs and failures, there are other more strategic, long-term benefits of network monitoring in the enterprise. First, reducing the workload of IT staff and making them more efficient is an important benefit. The automation and intelligence of network monitoring tools enable an IT staff to spend less time searching for the source of smaller outages, and this time savings can add up over the course of a year. If an IT administrator spent an average of four hours a week looking for smaller outages that amount to 10% of their work time, this means 10% of their annual salary too. Think of the time and money saved (or better used). When troubleshooting a network issue, most of the time is spent trying to find where the problem is vs. actually fixing the problem. Network monitoring helps IT to find where the problem is faster so that things can get fixed faster, leading to better network uptime and a more efficient IT staff.
A survey by network monitoring firm Paessler polled 648 of its customers in 2015, and 64% reported saving “significant” to “exceptional” amounts of time using network monitoring, while 24% reported saving “some” work time. Also, 87% reported network monitoring increased the general reliability of their IT system.
Along with the time savings and efficiency of IT spending less time on troubleshooting minor issues, there is the benefit of long-term IT optimization. An IT department that is not spending time on routine problems can now spend more time on strategic projects that can make a positive impact on the business. For example, maybe a new ERP upgrade takes four months instead of six months because IT staff is not spending as much time on smaller issues. Think of the positive impact to a business to have a major upgrade or implementation completed two months faster than normal. At the same time, IT staffers may be more satisfied with their work when they are spending less time on routine troubleshooting and more time on strategic projects. This satisfaction may lead to better employee retention and help to attract new IT talent as well.
In addition, network monitoring allows you to do long-term data collection to analyze how and where hardware and bandwidth resources are being utilized, so in the future routers, switches, servers, and other hardware can be acquired and used to meet exact requirements, helping to avoid excess costs due to unused resources. For example, rather than having a server running at 10% of capacity, you may be able to virtualize it to use more of its available capacity and get a better utilization. Or you could do continuous monitoring of a hard drive to predict when the capacity of the drive will be used up, allowing you to proactively plan for a replacement.
In all, while the numbers are often impressive, don’t just look at the numbers. Look beyond and factor in your own experience with outages that could have been prevented with network monitoring. Look at the “hard to quantify” benefits but see the value in their quality, as well as the long-term benefits of using resources better and being more efficient. This will help you to develop a more complete picture of the benefits of network monitoring.