I have the following python 3.12.4 code that inserts a JSON in a mongoDB deployed locally through a docker container using dockerhub' image "mongo:4.4.7"
myClient = pymongo.MongoClient("mongodb://%s:%s@%s" % (MONGO_USER, MONGO_PASS, MONGO_CONN), timeoutMS=10000)
myDB = myClient[CLIENT_NAME]
myCollection = myDB[COLLECTION_NAME]
the_json = '{"id": "json1","thedate": "2020-05-11T12:22:34.876Z"}'
myRecord = json.loads(the_json)
# replace str date with python' datetime object
myRecord["thedate"] = datetime.strptime(myRecord["thedate"], "%Y-%m-%dT%H:%M:%S.%fZ")
insertionResult = myCollection.insert_one(myRecord)
startDate = datetime(2020, 8, 12, 4, 21, 3, 923453)
mySearch = myCollection.find({"thedate": {"$gt": startDate}})
The above code returns records correctly.
The problem starts when using the very same python code against the very same dockerhub' image and the same python libraries in a kubernetes cluster. In such case, no records are returned. The curious part is that, in such cluster deployment, when I execute a "mySearch = myCollection.find()" it gets all the records in the DB, so there are no connectivity issues...
I have tried several types of date types to perform the search such as:
startDate = datetime(2012, 11, 2, 11, 10, 13, 123123, pytz.UTC)
startDate = datetime.now(tz=pytz.utc)- timedelta(days=7)
startDate = datetime(2013, 3, 12, 6, 54, 4, 123123)
startDate = datetime.now() - timedelta(days=4)
Just in case, the date of all containers:
MY PC DATE: lun 21 oct 2024 18:27:08 CEST+0200
DATE MONGODB RUN IN DOCKERS LOCALLY: Mon Oct 21 16:26:43 UTC 2024 +0000
DATE MONGODB RUN IN KUBERNETES: Mon Oct 21 16:29:24 UTC 2024 +0000
DATE PYTHON CODE KUBERNETES: Mon Oct 21 16:30:44 UTC 2024 +0000
As you can see, the date is stored with ISOdate, which seems correct to me. When I perform the following check using Mongo CLI the data type of the date stored in the DB seems also fine:
> date_obj = db["collection-name"].findOne({ thestr: 'exampleFile' }).thedate
ISODate("2024-10-15T11:59:59.111Z")
> date_obj.toString()
Tue Oct 15 2024 11:59:59 GMT+0000 (UTC)
Any clue about what could be happening? why is it working locally but not in kubernetes? Thanks in advance!!
Meanwhile, while you provide more information, let me post a snippet showing how I was able to have the Python client insert and read records from the MongoDB database, which is both deployed in the Kubernetes cluster.
Notice that the mongo URL is mongo-service.pythonclient.svc.cluster.local:27017". mongo-service is the name of the Service resource in Kubernetes used to connect to the mongo db instances at a stable endpoint.
You can kubectl apply -f <THE FILE BELOW>:
---
apiVersion: v1
kind: Service
metadata:
name: mongo-service
namespace: pythonclient
spec:
selector:
app: mongo
ports:
- protocol: TCP
port: 27017
targetPort: 27017
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mongo
namespace: pythonclient
spec:
serviceName: mongo-service
replicas: 1
selector:
matchLabels:
app: mongo
template:
metadata:
labels:
app: mongo
spec:
containers:
- name: mongo
image: mongo:4.4.7
env:
- name: MONGO_INITDB_ROOT_USERNAME
value: root
- name: MONGO_INITDB_ROOT_PASSWORD
value: foo
ports:
- containerPort: 27017
---
apiVersion: batch/v1
kind: Job
metadata:
name: pythonclient
namespace: pythonclient
spec:
template:
spec:
restartPolicy: Never
containers:
- name: pythonclient
image: python:latest
command:
- /bin/sh
- -c
- |-
cd /usr/app/src
pip install -r requirements.txt
python3 pythonclient.py
ports:
- containerPort: 8001
volumeMounts:
- mountPath: /usr/app/src
name: config
volumes:
- name: config
configMap:
name: pythonclient-config
---
apiVersion: v1
kind: ConfigMap
metadata:
name: pythonclient-config
namespace: pythonclient
data:
pythonclient.py: |
#!/usr/bin/env python3
##here goes the python code that save/reads from the mongoDB##
import pymongo
import json
from datetime import datetime
# docker run --rm -p 27017:27017 -e MONGO_INITDB_ROOT_USERNAME=root -e MONGO_INITDB_ROOT_PASSWORD=foo mongo:4.4.7
MONGO_USERNAME = "root"
MONGO_PASSWORD = "foo"
MONGO_URL = "mongo-service.pythonclient.svc.cluster.local:27017"
MONGO_CLIENT_NAME = "my-client"
MONGO_COLLECTION_NAME = "my-collection"
myClient = pymongo.MongoClient("mongodb://%s:%s@%s" % (MONGO_USERNAME, MONGO_PASSWORD, MONGO_URL), timeoutMS=1000)
myClient = pymongo.MongoClient("mongodb://%s:%s@%s" % (MONGO_USERNAME, MONGO_PASSWORD, MONGO_URL), timeoutMS=1000)
myDB = myClient[MONGO_CLIENT_NAME]
myCollection = myDB[MONGO_COLLECTION_NAME]
the_json = '{"id": "json1","thestr": "exampleStr","theint": 11,"thedate": "2024-10-15T11:59:59.953Z"}'
myRecord = json.loads(the_json)
myRecord["thedate"] = datetime.strptime(myRecord["thedate"], "%Y-%m-%dT%H:%M:%S.%fZ")
insertionResult = myCollection.insert_one(myRecord)
startDate = datetime(2024, 9, 24, 7, 51, 5, 953123)
documents = myCollection.find({"thedate": {"$gt": startDate}})
print()
print('===> showing documents:')
for doc in documents:
print(doc)
print('done')
requirements.txt: |
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
blinker==1.8.2
certifi==2024.8.30
cffi==1.17.1
click==8.1.7
dnspython==2.6.1
Flask==3.0.3
flask-swagger-ui==4.11.1
itsdangerous==2.2.0
Jinja2==3.1.4
joblib==1.4.2
MarkupSafe==2.1.5
minio==7.2.9
paho-mqtt==2.1.0
pycparser==2.22
pycryptodome==3.21.0
pymongo==4.10.1
python-crontab==3.2.0
python-dateutil==2.9.0.post0
pytz==2024.2
six==1.16.0
typing_extensions==4.12.2
urllib3==2.2.3
Werkzeug==3.0.3
You should see the following from kubectl logs -n pythonclient <THE JOB POD NAME>.
===> showing documents:
{'_id': ObjectId('6717bf86e93e55221ffc8ae9'), 'id': 'json1', 'thestr': 'exampleStr', 'theint': 11, 'thedate': datetime.datetime(2024, 10, 15, 11, 59, 59, 953000)}
done