Package size difference when using pip vs lambda layer

[lakshmi-speak]

When I install the library via pip the package size for version 1.26.0 is only 1.6MB but when I use the layer, it seems to be a lot bigger (causing my lambda function to go over the 250 MB limit). My function code is only about 11 MB. Is this expected?

[michaelbrewer]

You existing lambda project probably includes some of the common dependencies like boto3, fastjsonschema and aws-xray-sdk, so installing the pip for aws-lambda-powertools would only include the source code of 1.6 mb. You can check this just by doing a blank install on a new venv:

$ python3 -m venv .venv
$ source .venv/bin/activate
$ pip3 install aws-lambda-powertools
$ du -sh .venv
96M	.venv

Where as the lambda layer for powertools is always adds aws-xray-sdk, fastjsonschema, boto3 etc.. dependencies too so doubling up the size.

So you have some options here:

1. Create a custom layer with your dependencies + powertools
2. Create a custom later with our dependencies + powertools minus boto3 (and using the existing runtime)
3. Not using the powertools layer and just packaging your lambda like before.

In the future some of this will be also resolved by an issue for boto3 to reduce it's size (boto/botocore#2365) and there is a powertools issue to remove boto from the lambda layer. (see #1164)

[heitorlessa]

Closing as we documented in v2 that all dependencies are optional and Lambda Layers are optimized to get to 2MB with all deps.

https://docs.powertools.aws.dev/lambda/python/latest/#lambda-layer