What is Amazon Elastic File System?
Amazon Elastic File System (Amazon EFS) is a simple, server less, set-and-forget elastic file system that transfers the file data without installing or managing storage. It is compatible with AWS services as well as on-premises resources, and it is designed to scale to petabytes on demand without interfering with applications.
Amazon EFS supports the Network File System Version 4 (NFSv4.1 and NFSv4.0) protocol, so the applications and tools you use today are compatible with Amazon EFS. Multiple compute instances including Amazon EC2, Amazon ECS, and AWS Lambda can simultaneously access an Amazon EFS file system, providing a common data source for workloads and applications running on more than one compute instance or server.
Use cases for EFS:
An EFS is suitable for the following use cases:
- Web serving and content management
- Enterprise application usage
- Media and entertainment
- Shared and home directories
- Database backups
- Developer and application tools
- Container storage
- Big data analytics
- Other applications where you need to connect a common data source to a single server or multiple servers.
KEY FEATURES:
- Fully Managed:
Amazon EFS is a fully managed service that provides NFS shared file system storage for Linux workloads. It’s simple to create and configure file systems with Amazon EFS. You don’t have to worry about managing file servers or storage, updating hardware, configuring software, or performing backups. Create a fully managed file system in seconds with the AWS Management Console, the AWS Command Line Interface (CLI), or an AWS SDK.
- Highly accessible and long-lasting:
Amazon EFS is designed for 99.999999999 percent (11 9s) durability and up to 99.99 percent (4 9s) availability. For file systems using Standard storage classes, Amazon EFS stores every file system object (that is, a directory, file, and link) redundantly across multiple Availability Zones (AZs). When you choose Amazon EFS One Zone Storage Classes, your data is redundantly stored within a single AZ. Amazon EFS is designed to withstand concurrent device failures by quickly detecting and repairing any lost redundancy. Furthermore, you can access a file system that uses Standard storage classes concurrently from all AZs in the AWS Region where it is located. In other words, you can design your application to fail over from one server to another. In other words, you can design your application to failover from one AZ to another in the region for maximum application availability. Mount targets for all Amazon EFS storage classes are designed to be highly available within an AZ.
- Data Protection:
Amazon EFS Replication:
EFS Replication allows you to replicate your file system data to another AWS Region or within the same Region in a few clicks, without the need for additional infrastructure or a custom process to monitor and synchronize data changes. Organizations in regulated industries are frequently subject to compliance requirements that require secondary data copies to be stored hundreds of miles away from the original. Amazon EFS Replication automatically and transparently replicates your data to a second file system in the region or AZ of your choice. You can enable replication on an existing file system using the Amazon EFS console, AWS CLI, and APIs. EFS replication is continuous and designed to provide a recovery point objective (RPO) and a recovery time objective (RTO) of minutes, allowing you to meet compliance and business continuity objectives.
You can also configure your destination file system separately from your source file system. Independent of the source file system, you can configure the destination file system’s lifecycle management policy, backup policies, provisioned throughput, mount targets, and access points. For example, you can reduce the storage costs of the destination file system by enabling EFS Lifecycle Management with a shorter age-off policy (such as 7 days) than the source file system’s age-off policy (such as 7, 14, 30, 60, or 90 days). You can also replicate from an Amazon EFS Standard storage class source file system to an Amazon EFS One Zone storage class destination file system, and vice versa.
AWS Backup:
AWS Backup is a fully managed backup service that allows you to easily centrally manage and automate backups of your Amazon EFS file systems, eliminating the need for expensive custom solutions and manual processes. AWS Backup goes beyond backing up EFS by centralizing data backup across other AWS services in the cloud and on-premises. As applications migrate to the cloud, their data can become dispersed across multiple services, making it difficult to manage and consolidate backup activity without writing custom scripts and performing manual tasks. You can centrally configure and audit AWS resources, automate backup scheduling, set retention policies, and monitor backup activity with AWS Backup.
Storage Classes:
Amazon EFS provides Standard and One Zone storage classes for both frequently accessed and infrequently accessed files. The performance of the Standard and One Zone storage classes has been optimized to deliver consistent low latencies. Amazon EFS Standard-Infrequent Access (EFS Standard-IA) and Amazon EFS One Zone-Infrequent Access (EFS One Zone-IA) storage classes are cost-effective for files accessed infrequently. Begin saving on storage costs with EFS Lifecycle Management and an age-off policy (7, 14, 30, 60, or 90 days). You can also set a policy to automatically move accessed files from the cost-optimized infrequent access storage classes to the performance-optimized storage classes if they are accessed.
With EFS Lifecycle Management policies enabled, you can automatically move files between EFS Standard and EFS Standard-IA storage, or between EFS One Zone and EFS One Zone-IA storage, saving up to 92% on storage costs.
Using the industry-accepted estimate that 20% of data is actively used and 80% is infrequently accessed, you can store your files on Amazon EFS at an effective price of $0.043/GB-month (20% $0.16/GB-month1 for files stored on Amazon EFS One Zone + 80% $0.0133/GB-month1 for files stored on EFS One Zone-IA = $0.043/GB-month) or at an effective price of $0.08/GB-month (20% $0.30/GB).
- pricing in US East (N. Virginia) Region; assumes 80% of your storage is in EFS One Zone-IA
- pricing in US East (N. Virginia) Region; assumes 80% of your storage is in EFS Standard-IA
| Region | Effective storage price ($/GB-Mo) – One Zone* | Effective storage price ($/GB-Mo) – Standard** |
| US East (N. Virginia) | $0.043 | $0.08 |
| US West (Oregon) | $0.043 | $0.08 |
| Africa (Cape Town) | $0.054 | $0.10 |
| Asia Pacific (Seoul) | $0.047 | $0.09 |
| Canada (Central) | $0.047 | $0.09 |
| Europe (Ireland) | $0.046 | $0.09 |
| AWS GovCloud (US-East) | $0.056 | $0.11 |
Security and Compliance:
Control network access to your file systems with Amazon Virtual Private Cloud (VPC) security group rules, and application access to your file systems with AWS Identity and Access Management (IAM) policies and Amazon EFS Access Points. Amazon EFS meets many eligibility and compliance requirements, allowing you to meet your regulatory obligations. Click here to see a list of Amazon EFS compliance programs.
Scalable performance:
Amazon EFS is intended to provide the throughput, I/O operations per second (IOPS), and low latency required for a wide range of workloads. Throughput and IOPS scale as your file system grows and can burst to higher throughput levels for short periods of time, supporting the unpredictable performance needs of file workloads. Amazon EFS supports performance of over 10 GB/second and more than 500,000 IOPS for the most demanding workloads.
Elastic and scalable:
Storage capacity is elastic with Amazon EFS, growing and contracting automatically as you add and remove files to provide storage capacity to your applications as needed. Provisioning is not required with elastic capacity, and you are only charged for what you use. Amazon EFS is intended to be highly scalable in terms of both storage capacity and throughput performance, scaling up to petabyte scale and allowing massively parallel access from EC2 instances. Throughput and IOPS scale as your file system grows, and file operations have consistent low latencies.
Encryption:
Amazon EFS offers a comprehensive encryption solution for both stored data and data in flight. The AWS Key Management Service (AWS KMS) manages encryption keys for data at rest, eliminating the need to build and maintain a key management infrastructure. Data in Transit Encryption uses industry-standard Transport Layer Security (TLS) to secure network traffic without requiring any changes to your applications. More information about encrypting file system data can be found in the encryption user documentation.
- Data Transfer:
AWS DataSync:
AWS DataSync is a managed data transfer service that makes moving data between on-premises storage and Amazon EFS easier and faster. DataSync allows you to transfer active datasets over the internet or AWS Direct Connect at up to ten times the speed of open-source tools, without requiring you to modify your applications or write to an API. The service can be used for one-time data migrations, ongoing workflows with periodic synchronization, or replication for data protection and recovery. Many tasks known to slow down migrations or burden IT operations, such as infrastructure management, encryption, data validation, and data transfer orchestration, are handled automatically by DataSync.
AWS Transfer Family:
AWS Transfer Family offers fully managed file transfer support directly into and out of Amazon EFS. Transfer Family supports Secure File Transfer Protocol (SFTP), File Transfer Protocol over SSL (FTPS), and File Transfer Protocol (FTP), allowing you to easily migrate your file transfer workflows to AWS.
Getting started with Amazon Elastic File System:
There are four steps that you need to perform to create and use your first Amazon EFS file system:
• Create your Amazon EFS file system.
• Create your Amazon EC2 resources, launch your instance, and mount the file system.
• Transfer files to your EFS file system using AWS DataSync.
• Clean up your resources and protect your AWS account.
Step 1: Create your Amazon EFS file system
To create your Amazon EFS file system, follow these steps.
- Go to https://console.aws.amazon.com/efs/ to access the Amazon EFS Management Console.
2. To open the Create file system dialog box, select Create file system.
3. Optional: Give your file system a name.
4. Choose your VPC or keep it set to your default VPC for a Virtual Private Cloud (VPC).
5. Choose one of the following for availability and durability:
- To create a file system that uses standard storage classes, use Regional. Standard storage classes replicate file system data and metadata across all AWS regions and availability zones. Regional has the highest availability and durability.
- One Zone is used to create a file system that makes use of One Zone storage classes. One Zone storage classes are less expensive than standard storage classes because they store file system data and metadata redundantly within a single availability zone.
Because EFS One Zone storage classes store data in a single AWS Availability Zone, data stored in these storage classes may be lost if a disaster or other fault affects all copies of the data within the Availability Zone, or if an Availability Zone is destroyed due to disasters such as earthquakes and floods.
If you select One Zone, select the Availability Zone in which you want the file system to be created, or leave the default setting.
6. Select Create to create a file system with the following service recommendations:
- Automatic backups are enabled; for more information, see Backing up and restoring Amazon EFS file systems with AWS Backup.
- Mount targets are created by Amazon EFS with the following settings:
- A mount target is created in each Availability Zone in the AWS Region where the file system is created for file systems that use Standard storage classes. A single mount target is created in the Availability Zone you specify for file systems that use One Zone storage classes.
- Located in the VPC you chose’s default subnets.
- Using the VPC’s default security group – After the file system is created, you can manage security groups.
For more information, visit Managing file system network accessibility.
- General Purpose Performance Mode: See Performance Modes for more information.
- Bursting throughput mode (see Throughput modes for more information)
- enabling data-at-rest encryption with your default key for Amazon EFS (
aws/elasticfilesystem)-for more information, see Encrypting data at rest. - Lifecycle Management- When Amazon EFS creates a file system, the following lifecycle policies are applied:
~Transition into IA set to 30 days since last access
~Transition out of IA set to On first access
See Amazon EFS lifecycle management for more information.
With the exception of availability and durability, encryption, and performance mode, you can customize the file system settings after you create them.
Choose Customize if you want to create a file system with a customized configuration. See Creating a file system with custom settings using the Amazon EFS console for more information on creating a file system with customized settings.
7. The File Systems page is displayed, with a banner across the top displaying the status of the file system you created. When the file system becomes available, a link to the file system details page appears in the banner.
Step 2: Create your EC2 resources and launch your EC2 instance
In this step, you will create a new Amazon EC2 instance running Amazon Linux 2 and configure it to mount the EFS file system you created in Step 1.
Unless you already have one, you must create a key pair before you can launch and connect to an Amazon EC2 instance. You can launch your EC2 instance after creating a key pair in the Amazon EC2 Console.
- To create a key pair
To create a key pair, follow the steps Setting up with Amazon EC2 in the Amazon EC2 User Guide for Linux Instances. You do not need to create a new key pair if you already have one. For this exercise, you can use your existing key pair.
To launch the EC2 instance and mount an EFS file system
- Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
- Choose Launch Instance.
- In Step 1: Choose an Amazon Machine Image (AMI), find an Amazon Linux 2 AMI at the top of the list and choose Select.
- In Step 2: Choose an Instance Type, choose Next: Configure Instance Details.
- In Step 3: Configure Instance Details, provide the following information:
- Leave Number of instances at one.
- Leave Purchasing option at the default setting.
- For Network, choose the entry for the same VPC that you noted when you created your EFS file system in Step 1: Create your Amazon EFS file system.
- For Subnet, choose a default subnet in any Availability Zone.
- For File systems, make sure that the EFS file system that you created in Step 1: Create your Amazon EFS file system is selected. The path shown next to the file system ID is the mount point that the EC2 instance will use, which you can change.
- The User data automatically includes the commands for mounting your Amazon EFS file system.
- Choose Next: Add Storage.
- Choose Next: Add Tags.
- Name your instance and choose Next: Configure Security Group.
- In Step 6: Configure Security Group, set Assign a security group to Select an existing security group. Choose the default security group to make sure that it can access your EFS file system.
- Choose Review and Launch.
- Choose Launch.
- Select the check box for the key pair that you created, and then choose Launch Instances.Once the EC2 instance is created and becomes available, it will be mounted to your EFS file system. At this point, you will be able to transfer files to your EFS file system.
Step 3: Transfer files to Amazon EFS using AWS DataSync
After you’ve set up a working Amazon EFS file system, you can use AWS DataSync to transfer files from another file system to Amazon EFS. AWS DataSync is a data transfer service that simplifies, automates, and speeds up data movement and replication between on-premises storage systems and AWS storage services via the Internet or AWS Direct Connect. AWS DataSync can transfer file data as well as file system metadata like ownership, timestamps, and access permissions.
Before you start,
we’ll assume in this step that you have the following:
a source NFS file system from which you can transfer files. This source system must be reachable via NFS Version 3, 4, or 4.1. On-premises data center file systems, self-managed in-cloud file systems, and Amazon EFS file systems are all examples of file systems.
- A destination Amazon EFS file system to which files should be transferred. Create an Amazon EFS file system if you don’t already have one. See Getting Started with Amazon Elastic File System for more information.
- Your server and network are compatible with AWS DataSync. See the AWS DataSync requirements for more information.
To use AWS DataSync to transfer files from a source location to a destination location, do the following:
- Download and deploy an agent in your environment and activate it.
- Create and configure a source and destination location.
- Create and configure a task.
- Run the task to transfer files from the source to the destination.
To learn how to transfer files from an existing on-premises file system to your EFS file system, see Getting Started With AWS DataSync in the AWS DataSync User Guide. To learn how to transfer files from an existing in-cloud file system to your EFS file system, see Deploying the AWS DataSync Agent as an Amazon EC2 Instance in the AWS DataSync User Guide and the Amazon EFS AWS DataSync In-Cloud Transfer Quick Start and Scheduler.
Step 4: Clean up resources and protect your AWS account
This guide includes walkthroughs that you can use to learn more about Amazon EFS. You can use the resources you created and linked to in this Getting Started exercise in those walkthroughs before performing this clean-up step. See Walkthroughs for more details. After you’ve finished the walkthroughs, or if you don’t want to go through them, take the following steps to clean up your resources and protect your AWS account.
To clean up resources and protect your account
- Connect to your Amazon EC2 instance.
- Unmount the Amazon EFS file system with the following command.
$ sudo umount efs - Open the Amazon EFS console at https://console.aws.amazon.com/efs/.
- Choose the Amazon EFS file system that you want to delete from the list of file systems.
- For Actions, choose Delete file system.
- In the Permanently delete file system dialog box, type the file system ID for the Amazon EFS file system that you want to delete, and then choose Delete File System.
- Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
- Choose the Amazon EC2 instance that you want to terminate from the list of instances.
- For Actions, choose Instance State and then choose Terminate.
- In Terminate Instances, choose Yes, Terminate to terminate the instance that you created for this Getting Started exercise.
- In the navigation pane, choose Security Groups.
- Select the name of the security group that you created for this Getting Started exercise in Step 2: Create your EC2 resources and launch your EC2 instance as a part of the Amazon EC2 instance Launch Wizard.
- For Actions, choose Delete Security Group.
- In Delete Security Group, choose Yes, Delete to delete the security group you created for this Getting Started exercise.
Advantages of using an EFS:
Amazon EFS is adaptable. That is, as folders and files are added or removed from the system, its storage capacity can be automatically scaled up (to add more storage) or down (to shrink storage capacity). This is a significant advantage over traditional storage solutions in that capacity can be added or removed without affecting users or applications.
Importantly, EFS storage is indefinite. When data is attached to an AWS compute instance, it does not disappear when the instance is restarted.
Disadvantages of Using an EFS:
There are a few limitations to Amazon EFSs:
There are no Windows instances. AWS Windows EC2 instances do not support Amazon EFSs. EFS volumes can only be used with non-Windows instances that support NFS volumes, such as Linux.
There are no system boot volumes. Amazon EFS volumes are also ineligible for use as system boot volumes. Elastic Block Store (EBS) volumes must be used to boot AWS EC2 instances. With one exception, EBS volumes are similar to EFS volumes. EBS volumes can only be linked to a single EC2 instance or server, whereas EFS volumes can be linked to multiple EC2 instances and on-premises resources.