Editing
Psycopg2 layer
Jump to navigation
Jump to search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
==Background== Postgres instance connectivity is a critical component of our current pipeline. This library however is not available by default in AWS Lambda which is quite a PITA. ==Current version== As at August 2024, there are two psycopg layers available:<ref>Note that <code>psycopg2</code> is a legacy term that refers to major version 2 of the library. Version 3 was released in October 2021, and is called as <code>psycopg</code> instead. Unfortunately this syntax is used a bit interchangeably for historic reasons, so double check to avoid any confusion.</ref> {| class="wikitable" ! Layer name || Psycopg version || Python runtime |- | psycop9 || 1.3.8 || 3.9 |- | 20240828_psycopg2<ref>There are multiple versions - this relates to the latest version only.</ref> || 3.2 || 3.12 |} Obviously layer <code>psycop9</code> is more or less EOL and should not be implemented in new functions.<ref>Existing functions should also be transitioned from this as soon as feasible.</ref> Layer <code>20240828_psycopg2</code> is the most current available at the time of writing and should be used by default for any Postgres connection. ==Updating/making new layers== For psycop specifically, but also generally, the following steps are necessary to make a functional layer for use in AWS Lambda: # Create a new folder and navigate to it # Create <code>requirements.txt</code> specifying anything necessary. For psycopg2: <code>echo 'psycopg2-binary' > requirements.txt</code> # Install required library locally using the following filepath exactly: <code>pip install -r requirements.txt -t python/lib/python3.12/site-packages/ --upgrade</code>.<ref>Python version in this example should be amended depending on version available on your local machine AND the intended Lambda runtime.</ref> ## The filepath /lib/python3.12/site-packages/ is super duper important!! # Zip up the dir you just made: <code>zip -r amazing.zip python</code> # $$$ Above instructions/examples for linux only - no idea how to do any of this on Windows or Mac. ==TODO== * Figure out how to catch new [https://www.psycopg.org/articles/tag/news/ psycopg releases] and upgrade layers accordingly.<ref>Probably just security or LTS releases but yeah nah.</ref> ==References==
Summary:
Please note that all contributions to Kautepedia are considered to be released under the Creative Commons Attribution-NonCommercial-ShareAlike (see
Kautepedia:Copyrights
for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource.
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
Navigation menu
Personal tools
British English
Not logged in
Talk
Contributions
Log in
Namespaces
Page
Discussion
British English
Views
Read
Edit
Edit source
View history
More
Search
Navigation
Main page
Recent changes
Random page
Help about MediaWiki
Tools
What links here
Related changes
Special pages
Page information