Run in PyCharm - no problem; Run in Gitlab CI - ModuleNotFound - Why?

General discussion for off-topic subjects.
guyl
Junior Member
Posts: 1
Joined: Mon Feb 07, 2022 8:25 am

Run in PyCharm - no problem; Run in Gitlab CI - ModuleNotFound - Why?

Postby guyl » Mon Feb 07, 2022 8:48 am

Greetings,

First question on the forum.

https://stackoverflow.com/questions/71007297/python-importing-main-problem-module-exists-works-in-pycharm-gitlab-says-n is the post I sent to SO. [BTW - how to you create a link rather than the entire URL?]

Rewriting it here both for convenience and because I have tried other techniques.

Python version in PyCharm: 3.8.5
Python version in Gitlab CI: 3.10.0

PyCharm version information:
PyCharm 2020.3.5 (Community Edition)
Build #PC-203.7717.81, built on March 25, 2021
Runtime version: 11.0.10+8-b1145.96 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Windows 10 10.0
GC: ParNew, ConcurrentMarkSweep
Memory: 1957M
Cores: 8
Non-Bundled Plugins: Batch Scripts Support, CMD Support, com.andrey4623.rainbowcsv, com.chesterccw.excelreader, com.maxifom.pyenv-integration, net.seesharpsoft.intellij.plugins.csv, Docker, net.ashald.envfile


I have spent 1+ days reading PythonDoc, forums, SO, etc., using debug output statements, and I still have not figured it out.

Structure:

Code: Select all

engine-load-tests\
    |-src\
        |- __init__.py
        |- main.py
            |- on_locust_init()
        |
        |- win_perf_counters
                |- __init__.py
                |- main.py
                    |- main([])
    |- __init__.py
    |- .gitlab-ci.yml



Code:

Code: Select all

engine-load-tests\src\main.py:

    conf_file = os.environ['LOAD_TEST_CONF']
    database = environment.parsed_options.database
    if database is not None:
        global pid
        # pid = main([conf_file, database])
        import sys
        print(sys.path)
        print(os.environ['PATH'].split(os.pathsep))
        from src.win_perf_counters.main import main
        pid = main([conf_file, database])



Code: Select all

win_perf_counters:
    class WinPerf:
        def x:
            ...
        def y:
            ...

    def main(args):
        // code that I'm trying to call

    if __name__ == "__main__":
        import sys

        main(sys.argv[1:])


Granted that calling `main` from another module may not be proper design, but this works in PyCharm.

So I can't imagine I would need an __init__.py, since it works without that in PyCharm.

Here is what I get in GitLab CI:

Code: Select all

[2022-02-07 08:12:02,508] BuildAgent5/ERROR/root: Uncaught exception in event handler:
Traceback (most recent call last):
  File "C:\GitLab-Runner\builds\9ceD5xZA\0\qa\engine-automation\engine-load-tests\venv\lib\site-packages\locust\event.py", line 40, in fire
    handler(**kwargs)
  File "C:\GitLab-Runner\builds\9ceD5xZA\0\qa\engine-automation\engine-load-tests\src\main.py", line 194, in on_locust_init
    from src.win_perf_counters.main import main
ModuleNotFoundError: No module named 'src.win_perf_counters.main'


I don't think it has to do with PATH and PYTHONPATH, but FWIW -
When I run in PyCharm:

Code: Select all

PYTHONPATH
['C:\\git\\engine-load-tests', 'C:\\git\\engine-load-tests\\venv\\Scripts\\locust.exe', 'C:\\Python38\\python38.zip', 'C:\\Python38\\DLLs', 'C:\\Python38\\lib', 'C:\\Python38', 'c:\\git\\engine-load-tests\\venv', 'c:\\git\\engine-load-
tests\\venv\\lib\\site-packages', 'c:\\git\\engine-load-tests\\venv\\lib\\site-packages\\win32', 'c:\\git\\engine-load-tests\\venv\\lib\\site-packages\\win32\\lib', 'c:\\git\\engine-load-tests\\venv\\lib\\site-packages\\Pythonwin']

PATH
['C:\\git\\engine-load-tests\\venv\\Scripts', 'C:\\Python311\\Scripts\\', 'C:\\Python311\\', 'C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath', 'C:\\Program Files\\Microsoft MPI\\Bin\\', 'C:\\Windows\\system32', 'C:\\Wind
ows', 'C:\\Windows\\System32\\Wbem', 'C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\', 'C:\\Windows\\System32\\OpenSSH\\', 'C:\\Program Files\\dotnet\\', 'C:\\Program Files\\Microsoft SQL Server\\130\\Tools\\Binn\\', 'C:\\Program Fil
es\\Microsoft SQL Server\\Client SDK\\ODBC\\170\\Tools\\Binn\\', 'C:\\Program Files\\Git\\cmd', 'C:\\Program Files (x86)\\Microsoft SQL Server\\150\\DTS\\Binn\\', 'C:\\Program Files\\nodejs\\', 'C:\\ProgramData\\chocolatey\\bin', 'C:\\
Program Files\\Microsoft VS Code\\bin', 'C:\\Program Files (x86)\\Microsoft SQL Server\\150\\Tools\\Binn\\', 'C:\\Program Files\\Microsoft SQL Server\\150\\Tools\\Binn\\', 'C:\\Program Files\\Microsoft SQL Server\\150\\DTS\\Binn\\', 'C
:\\Program Files (x86)\\dotnet\\', 'C:\\Users\\guyl\\apache-jmeter-5.4.3\\bin', 'C:\\Program Files\\k6\\', 'C:\\Program Files (x86)\\STMicroelectronics\\STM32 ST-LINK Utility\\ST-LINK Utility', 'C:\\Python38', 'C:\\Python38\\Scripts',
'c:\\git\\engine-load-tests\\venv\\Scripts', 'C:\\Program Files (x86)\\Google\\Chrome\\Application', 'C:\\Program Files\\OpenSSL-Win64\\bin', 'C:\\Program Files (x86)\\PowerShell\\7\\', 'C:\\Program Files\\PowerShell\\7\\', 'C:\\Progra
m Files\\Docker\\Docker\\resources\\bin', 'C:\\ProgramData\\DockerDesktop\\version-bin', 'C:\\Program Files\\MySQL\\MySQL Shell 8.0\\bin\\', 'C:\\Users\\guyl\\AppData\\Local\\Microsoft\\WindowsApps', 'C:\\Program Files\\JetBrains\\PyCh
arm Community Edition 2020.3.3\\bin', 'C:\\Users\\guyl\\.dotnet\\tools', 'C:\\Users\\guyl\\AppData\\Local\\Programs\\Fiddler', 'C:\\Users\\guyl\\apache-jmeter-5.4.3\\bin', 'c:\\git\\engine-load-tests\\venv\\Scripts', 'C:\\Program Files
 (x86)\\Google\\Chrome\\Application', 'C:\\Users\\guyl\\AppData\\Local\\atom\\bin', 'C:\\Program Files (x86)\\Rico Suter\\NSwagStudio\\', '']



When I run in GitlLab CI:

Code: Select all

PYTHONPATH
['C:\\GitLab-Runner\\builds\\9ceD5xZA\\0\\qa\\engine-automation\\engine-load-tests', 'C:\\GitLab-Runner\\builds\\9ceD5xZA\\0\\qa\\engine-automation\\engine-load-tests\\venv\\Scripts\\locust.exe', 'C:\\Python310\\python310.zip', 'C:\\Python310\\DLLs', 'C:\\Python310\\lib', 'C:\\Python310', 'C:\\GitLab-Runner\\builds\\9ceD5xZA\\0\\qa\\engine-automation\\engine-load-tests\\venv', 'C:\\GitLab-Runner\\builds\\9ceD5xZA\\0\\qa\\engine-automation\\engine-load-tests\\venv\\lib\\site-packages', 'C:\\GitLab-Runner\\builds\\9ceD5xZA\\0\\qa\\engine-automation\\engine-load-tests\\venv\\lib\\site-packages\\win32', 'C:\\GitLab-Runner\\builds\\9ceD5xZA\\0\\qa\\engine-automation\\engine-load-tests\\venv\\lib\\site-packages\\win32\\lib', 'C:\\GitLab-Runner\\builds\\9ceD5xZA\\0\\qa\\engine-automation\\engine-load-tests\\venv\\lib\\site-packages\\Pythonwin']

PATH
['C:\\GitLab-Runner\\builds\\9ceD5xZA\\0\\qa\\engine-automation\\engine-load-tests\\venv/Scripts', 'C:\\Python310\\Scripts\\', 'C:\\Python310\\', 'C:\\Python39\\Scripts\\', 'C:\\Python39\\', 'C:\\Python38\\Scripts\\', 'C:\\Python38\\', 'C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath', 'C:\\Windows\\system32', 'C:\\Windows', 'C:\\Windows\\System32\\Wbem', 'C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\', 'C:\\Windows\\System32\\OpenSSH\\', 'C:\\Program Files\\dotnet\\', 'C:\\Program Files (x86)\\Microsoft SQL Server\\150\\DTS\\Binn\\', 'C:\\ProgramData\\chocolatey\\bin', 'C:\\Program Files\\Docker', 'C:\\Program Files\\Microsoft SQL Server\\Client SDK\\ODBC\\170\\Tools\\Binn\\', 'C:\\Program Files\\Java\\jdk1.8.0_211\\bin', 'C:\\Program Files\\Azure Data Studio\\bin', 'C:\\Program Files\\nodejs\\', 'C:\\tools\\dart-sdk\\bin', 'C:\\tools\\flutter\\bin', 'C:\\ProgramData\\chocolatey\\lib\\maven\\apache-maven-3.8.4\\bin', 'C:\\Program Files\\Git\\cmd', 'C:\\Users\\BuildAgent\\AppData\\Local\\Microsoft\\WindowsApps', 'C:\\Users\\BuildAgent\\.dotnet\\tools', 'C:\\Sysinternals', 'C:\\Users\\BuildAgent\\.dotnet\\tools', 'C:\\Users\\BuildAgent\\AppData\\Roaming\\Pub\\Cache\\bin', 'C:\\Program Files (x86)\\Rico Suter\\NSwagStudio\\', 'C:\\Program Files\\Oracle\\VirtualBox', 'C:\\Program Files (x86)\\Oracle\\VirtualBox']


TIA for any suggestions!

Return to “Whatever”

Who is online

Users browsing this forum: No registered users and 17 guests