[SOLVED] Chalice and Upgrading jq 1.3 to 1.6 in AWS Code Build

Chalice and Upgrading jq 1.3 to 1.6 in AWS Code Build

I have been trying to figure out how to update AWS CodeBuild preinstalled jq from 1.3 to 1.6 because of the --argjson argument that needs to be used. A simple apt-get does not work.

Solution

I have been trying to figure out how to update AWS CodeBuild preinstalled jq from 1.3 to 1.6 so I can use some additional functions needed for my DevOps. I did figure it out, so this is more of an answer instead of a question, but I thought I would share since I couldn't find anything online specifically for CodeBuild.

# !/bin/bash
pip install --upgrade awscli
aws --version

# Upgrade JQ so JSON substitution can work for layers
wget https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz
tar -xzf jq-1.6.tar.gz
cd jq-1.6
autoreconf -fi && ./configure --disable-maintainer-mode && make && make install || exit 1
jq --version

Now the reason for this solution was so I can update my Chalice config file with a new lambda layer generated by my CodeBuild. Here is the whole solution for CodeBuild for Chalice:

# !/bin/bash
pip install --upgrade awscli
aws --version
# Upgrade JQ
cd ..

# Upgrade JQ so JSON substitution can work for layers
wget https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.tar.gz
tar -xzf jq-1.6.tar.gz
cd jq-1.6
autoreconf -fi && ./configure --disable-maintainer-mode && make && make install || exit 1
jq --version
cd ../
cd src

# Create virtual environment and install requirements
pip install virtualenv
virtualenv /tmp/venv
. /tmp/venv/bin/activate
pip install -r requirements.txt
pip install snowflake-sqlalchemy

cd ../
## Layer update
mkdir -p lambda_layers/python/lib/python3.7/site-packages
# 1) install the dependencies in the desired folder
pip3 install  --upgrade snowflake-sqlalchemy -t lambda_layers/python/lib/python3.7/site-packages/.
# 2) Zip the lambda_layers folder
cd lambda_layers
zip -r snowflake_lambda_layer.zip *
# 3) publish layer
layerResults=$(
    echo aws lambda publish-layer-version \
        --layer-name fl-snowflake-lambda-layer \
        --compatible-runtimes python3.7 \
        --zip-file fileb://snowflake_lambda_layer.zip
)
echo "layer response: $layerResults"
layerARN=$(${layerResults} | jq '.LayerVersionArn')
echo "LayerARN: $layerARN"
cd ../
cd src

## Update Config
contents="$(jq --argjson arn ${layerARN} '.layers = [$arn]' .chalice/config.json)" && echo ${contents} && echo ${contents} > .chalice/config.json

# Unit tests
pip install -r requirements-test.txt
pip install chalice
export PYTHONPATH=.
pytest ./tests/ || exit 1

# Run Migrations
alembic -x stage=${stage} upgrade heads || exit 1

# Load Fixtures
echo ${stage}
python chalicelib/Fixtures/loadFixtures.py ${stage} || exit 1

# Package and deploy Chalice
# chalice deploy --stage ${stage}
chalice package --stage ${stage} /tmp/packaged
aws cloudformation package --template-file /tmp/packaged/sam.json --s3-bucket "${APP_S3_BUCKET}" --output-template-file transformed.yaml