Following this post set of posts on running Nextflow with AWS Batch:
I’m writing a small guide on how to make a custom AMI with a big enough size to run some actual data with.
All the instructions are in fact listed in the custom AMI Nextflow docs and this solution is there, but as I struggled once already, I prefer to document it for everyone and especially future me.
1. Choose AMI
: Search for and choose Amazon ECS-Optimized Amazon Linux AMI
in the AWS MArketplace
2. Choose Instance Type
: Choose a t2.micro
3. Configure Instance
: Leave default configuration4. Add Storage
: Input 500 GiB
or any other size, but 22 GiB
is definitively not big enough for actual real life data.5. Add Tags
: Do as you want6. Configure Security Group
: Just make sure you can connect to the EC2
Instance.7. Review Instance Launch
: Procure a key pair that you have so that you can access your InstanceActions
-> Instance State
-> Start
), it should be easy to findssh
command provided by AWS is wrong, you need to replace root
by ec2-user
sudo yum update
to update the systemawscli
using conda
, so install conda
first, and actually install wget
beforesudo yum install wget
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86\_64.sh
bash Miniconda3-latest-Linux-x86\_64.sh -p /home/ec2-user/miniconda
# You can either source .bashrc or specify the path to conda
source .bashrc
conda install -c conda-forge awscli
/etc/sysconfig/docker-storage
--storage-opt dm.basesize=500GB
(or specify any other size you prefer)sudo service docker restart
to restart the docker daemondocker info | grep -i base
and docker info | grep -i data
For all their support, thanks to:
Your email address will not be published