Setup an EC2 instance to run performance database setup scripts

Owned by Andrew Watkins
18-03-2020
1 min read

For performance testing we want a database configured with a large number of entities. This ensures that page loads, searches, matches etc take place with representative numbers of results.

A command-line interface script node x/perf/initPlatform.js can be used to populate the database with small, medium or large numbers or entities. options are xs, s, m, l, xl, xxl

The small options - xs, s and m can all be run on a local device. However, the l, xl and xxl will take multiple hours to run.

This page sets up an AWS EC2 instance that can run the script.

Prerequisites

  • A running EC2 instance. ( I used t2.xlarge )

  • SSH connection to the instance.

Installations

Git

sudo yum install -y git

Node

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh | bash

then close and reopen the terminal to load the new profile

nvm install node

MongoDB

A local database is used by the person generator. Also handy if you want to run voluntarily.

Create a repo entry

sudo vi /etc/yum.repos.d/mongodb-org-4.2.repo

insert this text

[mongodb-org-4.2] name=MongoDB Repository baseurl=https://repo.mongodb.org/yum/amazon/$releasever/mongodb-org/4.2/x86_64/ gpgcheck=1 enabled=1 gpgkey=https://www.mongodb.org/static/pgp/server-4.2.asc

Then run

sudo yum install -y mongodb-org

Start MongoDB

sudo systemctl start mongod

Services

Person Generator Service

This service manufactures fake people.

git clone https://github.com/RandomAPI/Randomuser.me-Node.git user cd user npm ci

Edit app.js line 68 to comment out the bandwidth throttling so we can create 1000s of people.

build it

npm run build

then run in the background

npm start &

you can view the service on localhost:3000

Install Voluntarily

git clone https://github.com/voluntarily/vly2.git cd vly2 npm ci

Create a .env file with the mongo connection url MONGODB_URI

vi .env

Run the script

node x/db/perf/initPlatform.js m --keep

first parameter is the size.

--keep will keep the existing database and add new values, without it the database will be wiped first.

Related articles