In today’s cloud orientated world, lots of File Servers lose the battle to modern solutions like Teams and Sharepoint. But what if these solutions don’t work for your company. For example, these files are not supported on those platforms or the applications working with the files don’t support accessing it from any other type of share than CIFS or SMB. Files are too big and accessed to regularly which cause latency or a too big demand on the internet connection?
Are you out of luck and cannot take advantage of cloud services for this? Maybe not yet.. Typically, for most of the organizations 80% of the data is never to almost never accessed. What if the clients and applications require that the hot data is available with low latency? We can use Azure File Sync to move the 80% of cold data to Azure Files. This way we save disk space on storage. If all the files are stored in Azure it is also possible to store them in Azure Backup! It is most likely cheaper and automatically offsite.
Let’s look at Azure Files and Azure File Sync!
What is Azure Files?
With Azure Files you have the ability to store files in Azure Storage accounts which can be presented as files shares that you can access over SMB. Microsoft put in a lot of effort the past year to enable NTFS permissions on this, and they have done a pretty good job in making it more usable now. They enabled AD integration so you can use NTFS permissions and groups now. And you can access the shares from remote offices and your datacenters with Azure Private Link. And last but not least? you can also use DFS-N (Distributed File Service Namespace) now.
At Splitbrain we always recommend the use of DFS-N in file servers. DFS-N gives you great flexibility in case of migrations or when you need to move files and folders around to other disks or servers.
What resources do we need?
You can use different storage accounts based on your needs. For example in the case of availability you can use Local Redundant Storage (LRS), Zone Redundant Storage (ZRS) or go global with Global Redundant Storage (GRS). And there is a performance point of view with regular or premium storage.
In Azure you pay for the resources you use. For Azure Files you can take a look at Azure Files pricing overview to calculate how much it will cost if you migrate your files to Azure Files. Keep in mind that not only storing the files in the share cost money. With accessing, listing and changing the files you will also be charged based on x amount of actions. In addition downloading the files from Azure Files to the client are charged as described here.
What is Azure File Sync?
With Azure File Sync you can extend your current file server to Azure in a tiered storage principle. You use your current file server as endpoint for your clients with all the current features there are today and offload the bulk of your data to Azure. You can configure the Azure File Sync agent to keep files newer than x amount of days on the file server per share or choose to not tier at all. This way most of the hot data is locally and only changes are sync from and to Azure.
With Azure Files, Azure File Sync and your file server you can have best of both worlds!
Note that you cannot use DFS-R and AFS Tiering on the same volume, they will bite each other.
What resources do we need?
With Azure File Sync we also require storage accounts to store the data. The same rules apply for the different types of storage accounts as described in the chapter Azure Files. On top of that we need a storage sync service to takes care of the synchronization and access of the data from the file server.
For Azure Files information you can view the pricing chapter for Azure Files above. On top of that we also need an Azure File Sync Service (or storage sync service) and that one is free if you only use one file server. When you have more you will be charged for the additional servers. Keep in mind that most of your data that is changing is on your file server. So your operational costs will be lower with Azure File Sync in compare to Azure Files.
Now let’s dive into that!
Azure Files and Azure File Sync use cases
To give you a better understanding of the possibilities with Azure Files and Azure File Sync we describe some example scenario’s below.
In this use-case the company moves all the files to Azure Files. Most of the companies keep the SMB port (445) closed to the internet. In addition, there are several ISPs that block SMB on their networks. To be able to move and access the files from the datacenter and/or office locations, we first need to setup a Site-to-Site VPN or an ExpressRoute. We also need our ADDS (Active Directory Domain Services) synced to AAD (Azure Active Directory) and enable ADDS authentication on the storage account for the file shares. The setup requires an Azure Private Link and Azure Private DNS to be able to resolve and access the Azure files shares over the ExpressRoute or VPN.
If your company is already using several resources in Azure and you already have an ExpressRoute or VPN connection you can leverage this.
We also need a Domain controller and DNS server in our datacenter/office and a server to host our DFS namespace. When we put all that in the mix and configure it the right way your users are able to access the file shares running in Azure.
User A accesses the DFS Share and browse the Marketing folder. When opening the file, it is downloaded from the storage account to his device and opened.
This scenario is great for small deployments, but this can get problematic for a number of reasons. For example when you have lots of users accessing the files. The files might be too large. These examples would have impact on the available internet bandwidth and could lead to other issues in the organization. The added latency could become a problem for some applications and the application experience becomes slow. If the internet connection is gone, so are the files. And there are more of these technical challenges to tackle when your environment is getting bigger. For these reasons Azure Files might not be the solution for your organization.
Single file server with Azure File Sync
We have a single file server running in your datacenter with for example 10 terabytes of storage. That takes up a lot of data on the underlying storage system and the backup.
For this setup we need very little configuration in Azure. An Azure subscription with a storage account and Azure File Sync Service is all that we need. In the scenario described above all clients connect directly over SMB to the storage account. In this setup all users use SMB to connect to the file server and if a file is not local, the agent pulls the parts of the file that are needed over SSL to the server. If DFS-N is in the mix, it is also easy to migrate to new smaller disks while uploading the data to Azure to save space on your file server and storage. You can enable deduplication to further lower the storage footprint. Beware that files are unduplicated when they land in the storage account, so we cannot save space there.
In this scenario user A accesses files from the IT folder in the DFS Share that are stored on the file server and delivered from local cache. The other file is from the marketing folder and not locally. The agent pulls the first bits to open the file to improve load time for the user while it continues to download the remaining bits. This way, Azure File Sync gives a performance advantage over the first scenario with Azure Files when working with big PDFs, PowerPoints or word documents when tiered files need to be downloaded from Azure Files.
If your company is already using several resources in Azure and you might have an ExpressRoute connection, you can take advantage of syncing over Express Route instead of over the internet and use private endpoints to further lock down access.
File server cluster with Azure File Sync
In some cases, a company is more dependent on the files being available at all times and have an highly available (HA) file server. If you require an HA file server and currently have a cluster? No problem! The Azure File Sync agent is cluster aware. All we need is a second agent added to the scenario above with the single server. This way we can offload the data in the file server cluster to Azure File Shares.
Multiple sites and regions use cases
Most of the Azure resources and that include Azure Files are bound to their region for various reasons. For example underlying infrastructure reasons like network latency between resources. If your company operates on more geographical locations, it might not work to centralize files to a single place or a single region for Azure Files. In order to use multiple regions we need additional resources in the second region. US users accessing their files in Europe is less efficient because of the large distance. We can use Azure Files and Azure File Sync to offload data to Azure to overcome these challenges.
Multi region with Azure Files
In the example below we have offices in Europe and the US. The users can access their files through their locally available DFS Namespace and even access files from other locations. All files are pulled from Azure files to the user device for accessing and editing.
Based on the setup, cross region traffic flows might be different. In the above example User C access a file in the Marketing US folder that is close to his office. User D access a file from the IT EU folder. The file from the Europe region is downloaded to his device and opened. Like in the first scenario all files come directly from the internet.
Multi region with Azure File Sync
As described in the scenario above with Azure Files, the setup with multiple locations is also possible with Azure File Sync. Users in both locations can access the files from their local file server and if the files are not available locally, the Azure File Sync agents pulls the bits from Azure Files transparently. When accessing files in the other region, like User C does, the same rules apply. If the file is locally available on the Europe file server, it’s presented directly to the user. If the file is not locally available, the agent of the file server in that region downloads the bits and provides it to the user so the file can be accessed.
Move to Azure IaaS with Azure File Sync
In some cases, companies move their file server to Azure IaaS. While the concept “Lift and Shift” looks plain and simple for file servers, it might not always be the answer because of disk efficiency, size and layout. Moving your file server 1-on-1 without optimizing it with cloud services can become a very cost inefficient solution. We have worked at projects with file servers containing 15TB of storage costing well over $2000 per month. When we optimized the fileserver with Azure File Sync, the costs dropped with more than 50%. The more storage, the bigger the savings.
Pro-tip: Azure Backup
Azure Backup is an Hybrid Backup solution on Microsoft Azure. With Azure Backup you are able to backup your VMs, SQL workloads and Files from your datacenter or hybrid datacenter. When using Azure Files and Azure File Sync you could leverage Azure Backup to further reduce datacenter storage costs and take advantage of the native integration of Azure Backup in Azure Files.
In the case of Azure Files you may no longer have the option to backup the files in your datacenter. Well you technically could, it is probably not very efficient… With Azure Files sync you could do a file level backup of your file server but that would initiate a download of all the files and we don’t want that to happen.
When going forward with a hybrid datacenter and moving files to the cloud, its important to also include the backup strategy in your design. When using Azure files it could be more cost efficient to use Azure Backup instead of you current backup solution.
To wrap things up, we go back to our initial question. Can you still move files to the cloud if you can’t use Teams or SharePoint.. Yes you can!
If you are looking at lowering the storage footprint or postpone a storage investment the above described scenarios might benefit you greatly. Start thinking about what an hybrid datacenter could do for you.
Although it might seem straightforward, setups like this can be quite complicated and specific features might not work as you expect, backup is not as simple now and DR brings new options to the table. Based on the design there could also be hidden costs that are not as obvious as plain storage and/or license costs? At Splitbrain were a happy to help you out if you are looking at a hybrid datacenter with Azure Files and/or Azure File Sync, just drop us an e-mail!