Discussion on: Dead Simple Python: Project Structure and Imports

[Ashley Hoff]

Hi!
Firstly great article. This has been one of the clearest examples of how it should be done. Thanks.

I am not sure whether this is an edge case, but I have a structure that looks like this:

generateandsend/
├── __init__.py
├── __main__.py
│
├── generatedata/
│   ├── __init__.py
│   └── generate_data.py
│ 
├── senddata/
│   ├── __init__.py
│   └── send_data.py
│ 
├── utilities/
│   ├── __init__.py
│   └── local_utilities.py
│ 
├── Readme.md
└── License.md

Both generate_data.py and send_data.py reference functions in local_utilities.py.

The issue I have, more often then not, I would be calling send_data.py or generate_data.py

I know that if I call either them specifically, I will need to add a reference to be able to import local_utilities.

Does this go against the general accepted practice? Would it be better to either separate them into different projects (I would like to keep all the code together) or use an argparser in __main__ and call the respective module using args?

Thanks
Ashley

[Ashley Hoff]

I also have one more question - if I wanted to include an ini/configuration file in a resources folder, how would I import it?

Thanks

[Jason C. McDonald]

I like to put all such non-code files in a project subdirectory (not a package) called resources, and then use the built-in package pkg_resources to access it.

For example, in my omission project, the module omission/game/content_loader.py needs to load the text file omission/resources/content/content.txt. I do that with...

import pkg_resources

class ContentLoader(object):

    def __init__(self):
        """
        Open the file and load the contents in.
        """

        # ...

        path = pkg_resources.resource_filename(
            __name__,
            os.path.join(os.pardir, 'resources', 'content', 'content.txt')
        )

        with open(path, 'rt', encoding='utf-8') as content_file:
            raw_content = content_file.read()

        # ...

Simple as that!

P.S. If you find yourself needing to access files outside of your project directory, say, in the user's home directory, I recommend the package appdirs.

[Ashley Hoff]

Again, thanks for the reply.

I've had a play with this one. Considering I am dealing with an ini file, it appears that configparser does what I want. This is the snippet I've come up with:

def conf():
    config = configparser.ConfigParser(converters={'list': lambda x: [i.strip() for i in x.split(',')]},
                                       allow_no_value=True)
    config.read('generateandsend/Resources/generateandsend.ini')
    section = config['test']
    string_a = section.get('StringA', None)
    string_b = section.get('StringB', None)

    return string_a, string_b

Is hard coding the relative path in that way frowned apon?

[Jason C. McDonald]

Is hard coding the relative path in that way frowned apon?

Most certainly, especially because you have to account for differences in path format between operating systems.

I'd recommend incorporating pkg_resources into your approach above.

def conf():
    config = configparser.ConfigParser(converters={'list': lambda x: [i.strip() for i in x.split(',')]},
                                       allow_no_value=True)
    path = pkg_resources.resource_filename(
            __name__,
            os.path.join(os.pardir, 'Resources', 'generateandsend.ini')
        )
    config.read(path)
    section = config['test']
    string_a = section.get('StringA', None)
    string_b = section.get('StringB', None)

    return string_a, string_b

I believe that will work? You'll have to check how config.read() handles an absolute path.

[Ashley Hoff]

Beautiful. Thanks. I had to massage it a little and remove os.pardir, as it was giving me a false directory on my windows machine (C:\tmp\generateandsend\..\Resources\generateandsend.ini).

The resultant path variable now looks like:

path = pkg_resources.resource_filename(__name__, os.path.join('Resources', 'dummy.ini'))

I just need to test this on my Linux box

Cheers again. Send the bill to...... 😉

[Jason C. McDonald]

Hi Ashley,

Sorry for the tremendous delay in reply. So, just to be clear, you're wanting to be able to call generate_data.py and send_data.py directly, and those are supposed to be able to import a module from elsewhere in the project?

If so, I would actually consider why you want to execute those modules directly. If you're simply wanting to be able to execute the two separately from the command-line, it may be worth fleshing out __main__.py to accept a command-line argument, so python3 -m generateandsend send or python3 -m generateandsend generate will execute what you want. That'll also be the easiest solution. That way, you're always executing the top-level package (generateandsend)

In fact, I'm not entirely sure off the top of my head how to get multiple projects to talk to one another within a shared directory! I know it has to do with PYTHONPATH, but I think that will necessitate more research on my part. ;)

[Ashley Hoff]

Thanks for replying (& no problem on the delay - we all have a life to live!).

I have thought about this more and agree - Why is it that I want to call them separately, where a parameter will suffice. So, I have abandoned the idea and gone with the python3 -m generateandsend generate approach

Cheers for the reply though. Appreciated.