Dynamic Provisioned & Dynamically Tiered Storage - Explained

Having looked around on internet for good references to these topics I found varied descriptions, so I've decided to create my own definition using a diagram (I learn better with Pictures).

Picture Reference, courtesy of myself:

Goodbye VMA for storage integration….Hello VASA API

Not long ago VMware announced that the VMA (Virtual Management Appliance) that is currently used as a conduit between the array and virtual centre would be wound up and replaced by a new storage API that will provide even tighter integration with storage arrays.

The API is called VASA – Vmware Aware Storage API

With this new level of integration VMware are touting that operations such as DRS (Dynamic Resource Scheduling) could be applicable for storage, where I/O intensive VM’s will be moved to higher performance LUNs/Datastores by way of array integration.

This causes a “catch 22” though as tiered storage environments will already being taking care of this at this back-end. However if your subsystem doesn’t support tiering, this API will be great for dynamic movement high I/O VM’s.

How to choose a storage vendor?

Recently I have noticed a number of discussions taking place on sites such as Linkedin under the “Storage Experts” and “Storage Processionals” section that ask for references on which vendor is best.

Reading the comments it is evident that there are some “die hard” vendor supporters out there that would rather sell their first born than stray to a different vendor.

With 7 years experience in the IT industry, I have seen my fair share of vendor "pre-sale" presentations and I must admit that it's easy to get sucked in by the impressive  powerpoint slides and marketing speak - each vendor will no doubt hold benefits for you or your organisation, but first you need to first ask a few key questions so the best possible decision can be made:

• Scalability - Current storage and projected storage requirements for up to 3 years
• Types of hosts/applications accessing the disk along with typical I/O requirements from each vendors host/application (read vs write, random or synchronous & spikes or consistent)
• Budget Range – remember that a SAN or NAS doesn’t just mean the subsystem, you will need:

o   Cabling (Fibre or Copper)

o   Switching (Fibre, Ethernet or Converged (FCoE))

o   Hosts Bus Adapters (HBA’s) for hosts

o   Multipathing Software for link redundancy
o   Management tools

• Existing infrastructure for possible leveraging – e.g. spare switches for an ISCSI solution
• The need for file level storage (NAS) or block storage (SAN) or both
• Return on investment (ROI) time
• Subsequent years of maintenance support
• Vendor presence in your area – are there others customer sites you can reference
• SLA's on support services
• VAAI Support (VMware API’s for Array Integration) – This allows performance gains by offloading I/O intensive tasks to the subsystem and is becoming increasingly important amoungst large and I/O intensive VMware environments
• Dynamic pooling and tiering capabilities – this can dramatically reduce your spindle count when going for performance

With these key items covered you should be well positioned for your vendor decision.

On a side note I would recommend not just to consider the “big” vendors as there are a number of promising storage start-ups appearing in the Market such as Tintri http://www.tintri.com/. These companies are beginning to gain momentum and can often provide a great solution for much less money – that is until they are consumed by one of the “big” vendors. 

Also, if you are really strapped for cash and are not too fussed on support services, there is always the open-source path:

• Free NAS http://www.freenas.org/
• Open Filer http://www.openfiler.com/

    This software installs itself as an operating system that is compatible on most x86 systems and literally turns a server into a subsystem where you can define ISCSI or FC targets on the local storage. I would suggest at minimum though that you have a hardware maintenance agreement on the server & sturdy RAID controllers for the local storage.

Also, as a side reference check out the magic quadrant from Gartner below, which highlights the vendors currently excelling in the storage industry:

Finally...Good Luck!

The Tale of Two V's - Virtual Desktop Infrastructure (VDI) & The Virtual Storage Platform (VSP)

Following on from my recent blog about the Virtual Storage Platform (VSP) from Hitachi Data Systems (HDS), I figured it would be a good idea to elaborate on how it will be deployed in a real life situation. 

The current client I work for is in the process of deploying Virtual Desktop Infrastructure (VDI). The product is called “VMware View” and they currently have around 200 of a planned 750 sessions deployed. 

Before we get in to the core storage aspects of this project, let’s ask ourselves some key questions:

What are the requirements for getting a VDI project from planning to execution?

VDI holds many merits, however the infrastructure requirements to run such an environment are substantial. One of the key ideas that gets thrown around when considering VDI is: “Great! We can throw away our current desktops and replace with Thin Clients which are less than half the size of a regular PC, have a lower administration overhead, have no moving parts and as a result draw less power”.

Thin Client Picture Reference, courtesy of http://www.hp.com/:

This is all well and good with desktop savings of approximately 60% if comparing a $1000 desktop to a $400 thin client and centralised administration, but this can soon be absorbed when deploying the back-end infrastructure.

This can include at a “high level”:

• Storage - SAN or NAS (We run SAN and are moving to a dynamically pooled, 3 tier solution)
• Storage Connectivity - Fibre Channel or Ethernet Switches (We are moving to 8GB FC)
• Network Connectivity (We are moving to 10GB Ethernet)
• Server Hosts - Blade or Server (We are using IBM 3850 X5’s with 7560 CPU’s (4 sockets with 8 cores each @ 2.26Ghz), 256GB RAM & 146GB SAS 2.5” local drives)
• Licensing – Depends on licensing agreement with VMware, but this isn’t cheap!
• Cooling & Power – Dependent on current environmentals
• Thin Client Terminals (We have deployed HP)

Another thing you cannot afford to discount from a project like this is user acceptance and experience, given that their beloved desktops will be disappearing. The change needs to seamless and ensure that no functionality or performance is lost. The best thing to do here is deploy a POC (proof of concept) that includes key people from around the organisation and have the acceptance spread organically.

In summary of this information, you can see the importance of carrying out detailed analysis around cost savings, initial outlay and user experience. Luckily for this particular client; the desktop fleet was reaching almost 5 years of age, existing infrastructure could be leveraged for a POC (Server Hosts, SAN & storage connectivity). A greenfield project is a whole different ball game, SHOW ME THE MONEY!!

How to build storage for a Virtual Desktop Infrastructure (VDI) project?

When I first approached this, I instinctively contacted the vendor for some ballpark I/O (input / output operations) values. No such luck – the technology was too new.

**NB: We started with View 4.0 and have gradually moved to 4.6 as the technology matured and was more widely accepted.

To begin with; a finger was licked and placed in the air…

Then, a number of RAID5 (4D+1P) 146GB 15k SAS drives were configured and connected using multi-path 4GB fibre channel links (HDS AMS2500).

We quickly saw the profile for VDI (under 4.0) and were able to make a call on what was required for the disk build. To cope with the high write load of around 80% compared to 20% read, a RAID10 (4D+4D) configuration was deployed. Other key statistics observed were a read cache hit of around 50-60% and 10-12 I/O’s per Windows XP SP3 VDI session.

(All statistics had around a 25% overhead added to ensure that a move to Windows 7 in the future was possible)

NB: To gain an understanding of the underlying VDI disk architecture, see the diagram below for reference, courtesy of http://www.myvirtualcloud.net/):

At this point after multiplying these figures across 750+ users, we realised that a 'generic' disk architecture wouldn't be enough and started talking about SSD drives for the replicas and the ability to dynamically pool and tier our storage solution for the other disks. These two pieces of technology not only reduce the number of spindles required but allow for huge performance scalability.

NB: See my blog entry here for more information on the topic of dynamic provisioning: http://cjrnz.blogspot.com/2011/03/vmware-disk-formats-dynamic.html . Tiering is the use of more than 1 disk type & RAID type per pool and then moving pages of data between the tiers according to how "hot" the data is. (Page sizes will be dependent on your storage vendor)

What about the Standard Operating Environment (SOE)?

You need to ensure your SOE (Gold Master) for VDI users is as refined as possible, with any 'extras' being deployed by VMware Thinapp.(The 'thinner' this is, the better your performance will be, not to mention the reduction in administration overheads)

Once your SOE is under control you can then create a Gold Master in VMware View which will be used as a source for your replica(s) and finally the linked clones.

Which Thin Client Should I use?

Most people think of a thin client as a “dumb terminal” and believe that not much thought or research is required in selecting one. WRONG!

Yes, the processing may have been moved to the server but you still need an easy to configure, scalable and manageable desktop unit. 

·         Key items:

• Central management console for firmware updates
• Easy to deploy configuration file
• Easy lockdown of terminal
• Operating System 
• VMware View Client Integration
• Hardware options (multi-screen users, pass-through of devices)
• Physical Size

What server components are required for VDI?


VDI Hosts run the same as hosts in a normal VSpher server environment, so depending on your VDI session workloads, you should specs the servers accordingly and consult the vendor if unsure. 
On top of all of this is the View Composer Software which is the 'heart' of VDI. This brings together the physical ESX hosts, "connection-broker" servers which direct internal+external session requests, "security" servers which field VDI requests from outside of your organisation (DMZ Bound) and finally the desktop virtual machines.

Summary

For this particular customer, VDI has been a great success.
By utilising the existing infrastructure to build a 'true' POC and scoping all new hardware for worst case scenarios, we were able to architect a high performing and highly scalable infrastructure model. This translates to a hugley positive user experience and this in itself measures the success of any VDI project!

Key Planning & Consideration Areas:

• Detailed Cost analysis around Hardware Costs
• Environmental Benefit Analysis
• Proof of Concept (POC) Testing
• Detailed Performance analysis from POC
• Establishing a 'solid' SOE

If you have any questions about the topics I have talked about, please get in contact as I would be more than happy to assist.

Technical Links:

http://www.vmware.com/products/view/features.html
http://en.wikipedia.org/wiki/Storage_virtualization

Hitachi Data Systems Virtual Storage Platform (VSP)

With the purchase order now received at the vendors end, I’ve decided to write an article about the new enterprise VSP storage array from Hitachi Data Systems (HDS).

Over the past few months I have been researching a number of different enterprise arrays with the hope of meeting stringent criteria:

• Green (not in colour but in terms of its carbon footprint – this means power draw, cooling and hardware density)
• Scalability
• Redundancy
• Ability to mix with existing arrays (non vendor specific)
• Support for mainframe connectivity (FICON point to point)
• Storage Tiering (Tiering of data across multiple storage technologies such as SSD, SAS and SATA so that high I/O is dynamically elevated to the correct tier)
• Thin Provisioning (Ability to only allocate storage that is actually used and reclaim overheads)
• VAAI (VMware Array Application Integration with Virtual Center). For those of you who aren’t aware what this means, it involves 3 different VMware API’s that ultimately improve efficiency:

1. Hardware Locking Offload – Ability to have more than one ESX host access and then write to a LUN/Datastore simultaneously, assisting in operations such as VMotion, creating new VM’s and deploying from template.
2. Write Same – When writing to a VMFS LUN/Datastore the data must first be zeroed out completely using a SCSI write command, this ensures integrity of data. By offloading the repetitive commands of identical zero blocks to the array, tasks such as formatting and reallocation are dramatically reduced in terms of I/O – up to 10x.
3. Fully Copy – Typically VM’s that are being vmotioned, cloned or even created the data must first be read by the host before being written back to the array. This is no longer the case as the API can now handle the task, reducing times by more than 50%.

**If you’d like to read a good article which references actual performance gains from the technology, go here: http://www.yellow-bricks.com/2011/03/24/vaai-sweetness/

 

The criteria above is specific to a customer’s requirements but I would imagine it to be at least 80% is relevant for most large corporates who employ SAN storage and are looking to create a totally dynamic and scalable storage solution.

As mentioned above I looked at a few different arrays such as the IBM V7000 and the NETAPP FAS3210. Now, I know a lot of you will be asking the question… where is the VMAX or VPLEX from EMC? Not this time unfortunately (I have a background in EMC SAN so I was disappointed not to have at least seen a demo).

Each of these arrays displayed impressive feature sets, however at the end of the day it was the VSP from HDS that took honors. Albeit this box isn’t the best looking with its green face-plates (image below), but its features have truly put it ahead of the class. Just to name a few that were stand outs:

• 2.5” 10K drives for improved array density (equivalent to 3.5” 15k). Up to 256 drives in your initial rack with controllers and 384 for each subsequent.
• VAAI functionality (first to market out of all vendors, which begs the question… where were you EMC, I thought you guys owned VMware???) http://www.theregister.co.uk/2011/02/08/hds_vaai_first/
• 6GB SAS switched back-end
• 8Gbps Fibre Channel Connectivity
• Ability to reverse the role of front end directors (FEDS) in to back end directors (BEDS)
• Priceless peace of mind knowing that Hitachi Data Systems employ disk from Hitachi Global Storage Technologies. End to end manufacturing has its advantages!
• Scales up to 1TB of flash

**NB: The unit still needs 8 x 32amp feeds (2 for each of the 4 x PDU’s)

With the array only a few weeks away from arriving on the floor it’s time to begin running fibre and finalising its initial uses. For this particular customer we will be rolling out its features in phases and the 1^st phases includes:

• Dynamic Tiering
• Dynamic Pooling
• FED to BED virtualization for existing arrays

In my next blog I will talk about the new found relationship that this array will have with Virtual Desktop Infrastructure (VDI) – 750 x Windows XP 32bit Desktops running on 6 vSphere 4.1 hosts with you guessed it… a VSP at the back end!

VSP Image Reference Courtesy of HDS.COM:

VMware Disk Formats & Dynamic Provisioning Of Storage

The other day whilst discussing dynamically provisioned storage, I came across 3 different disk formats used within VMware and thought they worth worth noting. But first, lets talk about Dynamic Provisioning....

Dynamic provisioning allows storage to be allocated to an application without it being physically mapped until it is used, this results in higher storage utilization within the organization. *Great idea...however it needs to be managed carefully so that you don't over allocate and find yourself with no disk left*. This style of provisioning feeds from a storage pool which involves the grouping of identical raid groups and striping the data across all of the spindles respectively - LUN's are then cut from this pool and presented accordingly.

Now that this is explained, the following benefits can now be mentioned:

• Adding physical storage non-disruptively as needed - adding additional raid groups to the pool(s)
• Transparently spread individual I/O workloads across multiple physical disks to balance workloads

Here is a great illustration of this:

**In my next blog I will expand further on this and enter in to the territory of dynamic tiering which involves multiple pools configured for different levels of I/O load.

Now, as promised here is my explanation of the 3 disk formats used within VMware:

Thin Format

When a thin format is configured, the .vmdk file is allocated only to the size required by the guest operating system. As write operations occur, the additional space is allocated and zeroed out in the .vmdk - the virtual disk grows out to the maximum allotted size as required.

Zeroedthick Format (VMware default)

When a zeroedthick format is configured, the .vmdk file is entirely allocated but the space is not “pre-zeroed" out. As the guest operating system writes to the .vmdk the space is zeroed as needed, but the .vmdk file itself does not grow in size. 

Eagerzeroedthick

When an eagerzeroedthick format is configured, the .vmdk file is the size allocated, and the unused space is zeroed out upon creation. As the guest operating system writes data to the .vmdk file the VMkernel does not need to write out zeros before the OS can access the storage. The result of this format is improved I/O latency but more back-end storage operations during the initial virtual disk creation. 

**This disk format is required for virtual machines configured with VMware fault tolerance - when this option is enabled the disk will convert to eagerzeroedthick. NB: Fault Tolerance works by continuously replicating a virtual machine on to a another ESX host within the cluster, when a failure occurs the replicated instance assumes the primary role with zero down time for the OS or application.

The Ethernet Fabric - Brocades answer to FCoE

Yesterday I read a white-paper by Brocade on their new technology concept called the "Ethernet Fabric". This technology allows FC protocol frames to travel within Ethernet packets (FCoE).

Fantastic!!! But what does this mean?

Currently the Fibre Channel protocol which is commonly used in SAN environments is "only" capable of speeds up to 8Ggps however its neighboring Ethernet protocol can now produce speeds of up to 10Gbps.

That's all well and good but you may be asking what the right solution is for you FCoE or iSCSI at 10Gbps?

Ultimately, this will depend on current infrastructure within the business along with what white-papers your technical architect has been exposed to. I believe the technology still needs time to mature as the capital expenses for a re-fit like this would be substantial, however the cost savings of only only maintaining one set of network architecture going forward are a big selling point.

To date I have worked with numerous iSCSI & Fibre Channel connected storage arrays and over the years have seen each protocol shine as the preferred solution for SAN connectivity. Personally my bets are on FCoE as most technology movements these days are based around integration, single console management and of course speed.

In today's market place there are two "main" vendors playing in the FCoE space - Cisco Nexus & Brocade VDX

The VDX series by Brocade comes loaded with their new Virtual Cluster Switching firmware, which allows up to 10 switches to be clustered together (fabric) and managed under a single interface.
With this firmware supporting the ability to use multiple ISL's (inter switch links) at once using self aggregation, the effective bandwidth is scalable to 600 10Gbit/sec.

In typical layer 2 switched Ethernet environments, 2 x protocols are required to ensure consistency and performance. These are: STP (Spanning tree Protocol) which is required to prevent switching loops along multiple ISL's and LAG's (link aggregation groups) to help achieve higher bandwidth between ISL's by pooling links together. All of this is left in the dust with "Ethernet Fabric" where almost any topology is supported.. Mesh, Ring, tree or core/edge!

The days of complete physical separation between LAN & SAN networks are numbered!

Brocade Fabric Changes via the GUI = null

The other day whilst making zoning changes on fabrics at a client, I encountered an issue where by any fabric changes would receive a "null" upon enabling the configuration (aliases + zoning groups).

The issue was quickly found within the Java Control Panel within my Windows control panel.

The root however was related to the clients local proxy settings.

Open the Java Control Panel and ensure that Direct Connection is enabled.

Navigation: General Tab to Network Settings Button to Direct Connection check box

HTC legend - Review

A little over a month ago I purchased an HTC Legend running Android 2.1 and now cant go without it. Not only was it a great deal ($49 per month over 12months with $550 value + 1.5GB data and the phone free) but the phone itself is great! 

These days it seems that if you don't own an iPhone there's something wrong with you and its just that attitude that pushes me to defy its trend setting ways. (Having said that I do own an iTouch so you could say my apple appetite has been served... excuse the pun)

From the moment I picked up the HTC box I noticed a change that closely follows the lead of Apple... cosmetically pleasing and streamlined packaging - its surprising how effective this is in marketing a product!

Next it was time to put the phone together (battery, SIM & included 2GB micro SD card) and straight away the phone feels great to hold. The combination
of aluminum casing which might I add is made from just one peice of aluminum + the ergonomic shaping/design doesn't make it feel like your holding a fragile glass brick. I wish I could say the same about the iPhone 4 but I cant.

The boot time is impressive where from power-off to making a call is around 30 seconds. The menu layout is excellent, although it does take a little
while to get used to some aspects - the most notable time saving GUI feature is the drop down bar to quickly view emails/txts/tweets whilst in the middle of anything.


My only quirks with this phone are:

*The lack of a front facing camera for taking advantage of apps such as Skype
*The limited "on-board" memory

Some great apps I've found so far using Android Market:

* 3G watchdog (monitor and set your data quotas)
* Angry Birds (extremely addictive but great for passing time)
* ColorNote (great for storing reminders/notes)
* Facebook for Android (I very rarely touch a computer for Facebook now)

Overall a great phone with no real regrets albeit the risk was low with a max commitment of $600 across the year. To aid me in making my decision I
used my favourite review site www.cnet.com.au

Now I need to decide whether I will delve into FroYo 2.2 for my handset, however I'm not yet convinced yet as many people to date are complaining about performance...watch this space though as I'm sure it'll get the better of me!

For more information on the HTC legend:
http://www.htc.com/www/product/legend/overview.html

For more reviews on the HTC Legend:

http://www.cnet.com.au/htc-legend-339301189.htm

Symantec Enterprise Vault

Last week I had the pleasure of meeting with two representatives from Symantec to talk about their Enterprise Vault archiving product.

As paper based systems slowly meet their demise, storage requirements and the need for scalable archiving have boomed within businesses both large and small.

Currently I am working on a large account that employs no means of file/email level archiving.

The solution is simply to buy more disk which in turns pushes the backup window out over the edge it is already clinging to.

Welcome Symantec to the stage!

Having already seen numerous software archiving solutions such as EMC Disk Xtender & CommVault/Simpana I wasn’t expecting to be dazzled by the technology however I did leave the meeting suitably impressed by their offering.

The product is driven via a 32bit application which can exist either as single node or as part of a cluster and is referred to as the Enterprise Vault server. This server will have access to different tiers of storage which are used to archive emails or files and ultimately reduce the physical disk requirement on the servers hosting the application data/functionality.

Security and policies are integrated with active directory so logical groupings are easily achieved and managed E.g. individual & group archives to mimic departmental structures.

The first item covered was Email archiving. This came in the form of a roll-out desktop client (Approx 4Mb .msi) which integrates itself with Outlook; connectivity is achieved through MAPI so there is no need for changes to the Exchange Server itself and syncing is done via a secret message within the users mailbox.

Interesting points:

*For mobile users a local cache of the Vault servers user contents is kept on the laptop

*Ability to search at multiple layers within Email: Message Body to Attachment to Sub Attachment (.zip contents)

*Attachments are kept as pointers within the message and automatically retrieved from Vault server storage at the request/double click of the user

*Has the ability to assist in email version migrations (e.g. Exchange 2007 --> 2010) by creating a transparent archive.

*“.PST search and destroy”... well not quite, but it can automatically go out and search/recover sprawled .PST files and consolidate in to the vault without interrupting the end-user experience.

The next was file archiving which was much simpler in comparison but used the same policy based retentions mechanisms. These retentions can be set at an organisation, group or individual level depending on compliance requirements of the business.

Interesting points:

*At the file system, files can be viewed as their actual size or pointer size when viewing the properties of the selected file.

*Ability to deliver files direct to the desktop or inflate on the server with a policy re-run required for delivery back to the vault.

*Restrictions can be placed on vault contents such as elimination of .mp3 files.

Recommendation for Single Node Deployment in virtual environment:

*Win 2KXXX

*2 x vCPU

*4Gb RAM (32bit limitation)

*Microsoft SQL backend for metadata

*SAS disk for index & SATA for archive (approx. 12% of archive is equal to the index size)

In summary I found Symantec Enterprise Vault to be a simple solution to an often complicated problem, with smooth integration and little of no interruption/training for the end user. When the project comes to fruition I will be sure to post my experiences here and provide a more in-depth review. 

For further information, please see the following data-sheet: http://eval.symantec.com/mktginfo/enterprise/fact_sheets/b-ev9_archiving_DS_20465026-1.en-us.pdf

In The Beginning...

After 6 years in the IT industry I've decided to start sharing my thoughts and experiences with technology. Even in this short time the IT landscape has changed dramatically with the upsurge of social media technologies such as Facebook and Twitter to the vast adoption of virtualization technologies such as VMware. Each of these technologies will forever change the way we use computers.

I still remember my first job in IT (*queue harp music to take us back in time*).

It was March 2004 and I was sitting in a computer lab working through my 2nd year University papers..CCNA curriculum if I remember rightly, when a lecturer handed out a job opportunity at the local council. Being more interested in drinking beer and living the student life I didn't think much of it and put it to the side. Later that night after a few quiet beers, something came over me and I decided to apply.

Two weeks later I had a floppy disk in my hand refurbishing old PC's and installing Windows 98 ready for local donations. After this stint I quickly moved to production PC builds and helpdesk fill-ins. A year later I was finishing my final exams and was offered a full time role as an "Information Technologist". Thanks to Steve and Peter for taking a chance on a scruffy IT student!

Now working as a consultant specialising in storage and infrastructure technologies I hope to provide some interesting and helpful insights in to the technology I deal with day to day.... Watch this space!