DEV Community: techocodger

A Small Code Renaming Example in Python

techocodger — Sat, 08 Jul 2023 05:27:54 +0000

I've recently gone to make some edits in a program, and immediately realised that my variable and function naming was poor and should be rectified before I make the next edits.

The general situation is in a part of the application Foldatry, in particular the part that does the hash-based matching of files by their content. In the process of that, it visits every file and calculates a hash by reading the entire file. This is a fairly standard way of finding files that have the exact same content, regardless of the filenames and timestamps.

What I would like to do, is to amend the program to let me have an option to do a faster process in which it first matches file by their filenames, sizes and timestamps. But first, some housekeeping seemed worthwhile.

Prefixes

As I'll be using them in this example, while not vital to make sense of the following, it may be useful to know my personal prefix codes are:

i = internal - where that seems useful to distinguish from one of similar name with a different prefix
v = var parameter - aka passed by reference (the "var" here comes from Pascal)
p = parameter - by passed by value
r = result - a value that will be quoted in a Python "return" statement
g = global - which in the context of Python means global to a module - I try to avoid needing this.

The Outer Call

Here is the outer layer of the call that does this.

def filedupetry_command( p_tree_paths, p_multi_log):
    # set up structures for Phase 1
    d_dct_size_lst_pfn = {} # dictionary of file sizes each with a list of pathfiles
    mtc_dct_hash_lst_tplfilinf = {} # dictionary of hashes 
    mtc_dct_file_selves = {} # dictionary of files with info about them
    # ---------------------
    # Phase 1 - tree traversal
    # Traverse the tree calculating and collecting hashes
    tpi = 0
    tpn = len(p_tree_paths)
    for i_tree_path in p_tree_paths:
        tpi = tpi + 1
        traverse_tree_add_hash( d_dct_size_lst_pfn, mtc_dct_file_selves, i_tree_path, p_multi_log)
    # ---------------------
    # Phase 2 - Hashes for files the same size
    build_hash_dct_from_Size_dct( mtc_dct_hash_lst_tplfilinf, d_dct_size_lst_pfn, p_multi_log)
    # ---------------------
    # Phase 3 - Analysis - make a size based dictionary of the hashes
    dct_size_lst_hash_multi_files = make_dct_size_lst_hashed_with_multi_files( mtc_dct_hash_lst_tplfilinf )
    # ---------------------
    return mtc_dct_hash_lst_tplfilinf, mtc_dct_file_selves, dct_size_lst_hash_multi_files

It was passed in a list of folders to process - i.e. a list of strings giving the paths. You can ignore all the p_multi_log references as that is for my logging system.

So actually the first thing that's poor about this code already: is that the name of a function it calls is quite misleading.
i.e. traverse_tree_add_hash because while that name was valid the first time I wrote it, I later changed the approach.
Actually now, that function builds just a dictionary of files keyed by their sizes - for the idea that only those of the same size should then be hashed for contents. However, this improvement is reperesented in the name of the local parameter being passed to it d_dct_size_lst_pfn being a dictionary keyed by size and having a list of path-filenames.

So, a better name would now be something like traverse_tree_make_sizes_dct

While we're here, let's note that the running of that function is just a first phase, with a second phase being to use that size-keyed dictionary to then build yet another resource keyed by the content hashes - by calling build_hash_dct_from_Size_dct

Even while we're looking at this outer part of code, there are some other names that I might now want to change.
Let's look at these three :

    d_dct_size_lst_pfn = {} # dictionary of file sizes each with a list of pathfiles
    mtc_dct_hash_lst_tplfilinf = {} # dictionary of hashes 
    mtc_dct_file_selves = {} # dictionary of files with info about them

At the time I probably named d_dct_size_lst_pfn with a d prefix to indicate it was "data" being assembled. Now, I would rename this to have a prefix of i to indcate that it is internal to this function, hence to become i_dct_size_lst_pfn

At the time I probably named mtc_dct_hash_lst_tplfilinf with a prefix of mtc to distinguish which layer of function it was in, with the "mtc" being a reference to that function context. That's not relevant here, and is only a hangover from when this function was a copy-and-adapt construction from earlier code. So this can be renamed to r_dct_hash_lst_tplfilinf

Similarly, mtc_dct_file_selves can be renamed to r_dct_file_selves to clarify that this will be built and then returned.

Ditto for

dct_size_lst_hash_multi_files which can become r_dct_size_lst_hash_multi_files

Hence, after applying those renamings the code looks like:

def filedupetry_command( p_tree_paths, p_multi_log):
    # set up structures for Phase 1
    i_dct_size_lst_pfn = {} # dictionary of file sizes each with a list of pathfiles
    r_dct_hash_lst_tplfilinf = {} # dictionary of hashes 
    r_dct_file_selves = {} # dictionary of files with info about them
    # ---------------------
    # Phase 1 - tree traversal
    # Traverse the tree calculating and collecting hashes
    tpi = 0
    tpn = len(p_tree_paths)
    for i_tree_path in p_tree_paths:
        tpi = tpi + 1
        traverse_tree_make_sizes_dct( i_dct_size_lst_pfn, r_dct_file_selves, i_tree_path, p_multi_log)
    # ---------------------
    # Phase 2 - Hashes for files the same size
    build_hash_dct_from_Size_dct( r_dct_hash_lst_tplfilinf, i_dct_size_lst_pfn, p_multi_log)
    # ---------------------
    # Phase 3 - Analysis - make a size based dictionary of the hashes
    r_dct_size_lst_hash_multi_files = make_dct_size_lst_hashed_with_multi_files( r_dct_hash_lst_tplfilinf )
    # ---------------------
    return r_dct_hash_lst_tplfilinf, r_dct_file_selves, r_dct_size_lst_hash_multi_files

Note that here we have glibly renamed a function call to be traverse_tree_make_sizes_dct but we will of course need to rename that where it is defined.

The Inner Call

Now let's look at the inner function, and what renaming could or should be done in there.

Actually, it is really two layers because there is a recursive function, with a non-recursive outer call.

Here is the outer call:

# recursively traverse a folder structure, deriving and storing hashes
# this is the outer non-recursive part
# currently the only difference is the use of a depth counter in the recursive call
# and that's only there to help with debugging
def traverse_tree_add_hash( d_dct_size_lst_pfn, folderselfs_d, p_tree_path, p_multi_log):
    i_lst_ncalls = [0]
    traverse_tree_recurse( d_dct_size_lst_pfn, folderselfs_d, p_tree_path, 0, i_lst_ncalls, p_multi_log)

in which I've left comments in, to show that these also need changing - as the function does not in fact do any "deriving and storing hashes"

So, as we've already covered, we want to rename this to traverse_tree_make_sizes_dct

# recursively traverse a folder structure, compiling a dictionary of sizes
# this is the outer non-recursive part
# currently the only difference is the use of a depth counter in the recursive call
# and that's only there to help with debugging
def traverse_tree_make_sizes_dct( v_dct_size_lst_pfn, v_dct_folderselfs, p_tree_path, p_multi_log):
    i_lst_ncalls = [0]
    traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_folderselfs, p_tree_path, 0, i_lst_ncalls, p_multi_log)

So what I've done there is to also do some internal variable renaming:

d_dct_size_lst_pfn becoming v_dct_size_lst_pfn
folderselfs_d becoming v_dct_folderselfs

Now let's look at the inner, recursive, function.

# recursively traverse a folder structure, deriving and storing hashes
def traverse_tree_recurse( d_dct_size_lst_pfn, d_pfn_stock, p_tree_path, p_tree_depth, p_lst_ncalls, p_multi_log):
    # d_hash_lst_tplfilinf is a dictionary being built, with hashes for keys, and lists of tuples for values
    # d_pfn_stock is a dictionary being built, with folderpaths for keys, and tuples for values
    # p_tree_path is the folder location now being accounted 
    p_lst_ncalls[0] += 1
    fldr_n = p_lst_ncalls[0]
    entries = explrtry.get_lst_sorted_os_scandir( p_tree_path )
    filcount = 0
    dircount = 0
    for entry in entries:
        if entry.is_file(follow_symlinks=False):
            # assemble specifics and derive hash
            pfn = im_osp.join(p_tree_path, entry.name) 
            size = entry.stat().st_size
            # make as a tuple
            n_t = tpl_FileInfo( pfn, entry.name, size, "")
            # now store it
            store_dct_size_lst_add_PathFile( d_dct_size_lst_pfn, n_t )
            store_dct_pfn_tpl_FileInfo( d_pfn_stock, n_t )
            filcount = filcount + 1
        elif entry.is_dir(follow_symlinks=False):
            # recursive call returning the hash of the subfolder
            new_subpath = im_os.path.join(p_tree_path, entry.name)
            traverse_tree_recurse( d_dct_size_lst_pfn, d_pfn_stock, new_subpath, p_tree_depth + 1, p_lst_ncalls, p_multi_log)
            # count subfolders and prepare for recursion further down
            dircount = dircount + 1
    return

Here is the plan of which things I want to rename:

d_dct_size_lst_pfn becomes v_dct_size_lst_pfn
d_pfn_stock becomes v_dct_pfn_stock
p_lst_ncalls becomes v_lst_ncalls - we'll say more about this later

Here is the code after the renaming has been done,

# recursively traverse a folder structure, deriving and storing hashes
def traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_pfn_stock, p_tree_path, p_tree_depth, v_lst_ncalls, p_multi_log):
    # d_hash_lst_tplfilinf is a dictionary being built, with hashes for keys, and lists of tuples for values
    # v_dct_pfn_stock is a dictionary being built, with folderpaths for keys, and tuples for values
    # p_tree_path is the folder location now being accounted 
    v_lst_ncalls[0] += 1
    fldr_n = v_lst_ncalls[0]
    entries = explrtry.get_lst_sorted_os_scandir( p_tree_path )
    filcount = 0
    dircount = 0
    for entry in entries:
        if entry.is_file(follow_symlinks=False):
            # assemble specifics and derive hash
            pfn = im_osp.join(p_tree_path, entry.name) 
            size = entry.stat().st_size
            # make as a tuple
            n_t = tpl_FileInfo( pfn, entry.name, size, "")
            # now store it
            store_dct_size_lst_add_PathFile( v_dct_size_lst_pfn, n_t )
            store_dct_pfn_tpl_FileInfo( v_dct_pfn_stock, n_t )
            filcount = filcount + 1
        elif entry.is_dir(follow_symlinks=False):
            # recursive call returning the hash of the subfolder
            new_subpath = im_os.path.join(p_tree_path, entry.name)
            traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_pfn_stock, new_subpath, p_tree_depth + 1, v_lst_ncalls, p_multi_log)
            # count subfolders and prepare for recursion further down
            dircount = dircount + 1
    return

Issues that Renaming Brings To Light

Now one of the benefits of doing this kind of renaming - other than making the code a little clearer for the next time it needs to be read and comprehended - is that the process brings to light various other things that should be re-done.

In this case, we bump into some things I did very early in the process of adapting from my generic programming experience over to how to do things in Python.

that the recursive calls should be sub-scoped to the outer call - i.e. the functions should be nested
that this allows a replacement of an ugly use of a list as a single value store

First let's look at the nesting. Here is a pseudo-brief of the functions as they are.

def traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_pfn_stock, p_tree_path, p_tree_depth, v_lst_ncalls, p_multi_log):
    return 

def traverse_tree_make_sizes_dct( v_dct_size_lst_pfn, v_dct_folderselfs, p_tree_path, p_multi_log):
    traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_folderselfs, p_tree_path, 0, i_lst_ncalls, p_multi_log)
    return

That happens to be how things get done when coding in a language that supports recursion but not nesting. An example of this is VBA.

As Python does support nesting (as indeed does Pascal which is my mental internal coding language) that means we can restructure like this:

def traverse_tree_make_sizes_dct( v_dct_size_lst_pfn, v_dct_folderselfs, p_tree_path, p_multi_log):
    def traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_pfn_stock, p_tree_path, p_tree_depth, v_lst_ncalls, p_multi_log):
        return 
    traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_folderselfs, p_tree_path, 0, i_lst_ncalls, p_multi_log)
    return

As presaged above, this allows a change to how one of the tracking variables is handled.

Now to be clear: as the code here was from my very early tackling of Python, when my inclination was to have a passed-by-reference scalar variable (i.e. just a number value) that would travel through the recursion to act as a progress counter. As Python doesn't support that idea - a scalar parameter when written to, creates a new local mutable instance - then I "got around" that by passing a list of just one value.

This makes for an ugly (in several senses) arrangement as see in this abstraction:

def traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_pfn_stock, p_tree_path, p_tree_depth, v_lst_ncalls, p_multi_log):
    v_lst_ncalls[0] += 1
    fldr_n = v_lst_ncalls[0]
    print( "Folders traversed: " + str( fldr_n) )
    return 

def traverse_tree_make_sizes_dct( v_dct_size_lst_pfn, v_dct_folderselfs, p_tree_path, p_multi_log):
    i_lst_ncalls = [0]
    traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_folderselfs, p_tree_path, 0, i_lst_ncalls, p_multi_log)
    return

Where the list-of-one-value has to be created in the outer layer, passed to the inner layer and carefully handled there - incremented and then passed to itself in recursion.

But with the nested version, we can dispense with the list-of-one-value and also not need to pass it as a parameter.

def traverse_tree_make_sizes_dct( v_dct_size_lst_pfn, v_dct_folderselfs, p_tree_path, p_multi_log):
    i_ncalls = 0
    def traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_pfn_stock, p_tree_path, p_tree_depth, p_multi_log):
        nonlocal i_ncalls
        i_ncalls += 1
        print( "Folders traversed: " + str( i_ncalls) )
        return 
    traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_folderselfs, p_tree_path, 0, p_multi_log)
    return

Do note that this requires one of Python's rare syntactic-only elements nonlocal to be deployed, to ensure that the required scoping of the variable i_ncalls is implemented across the function call depths. I simply didn't know about this when I originally wrote the code.

It's also a curious thing in Python syntax because it is only a guide instruction to the Python interpreter - no program action occurs at that point in the code. Also - in case it's not obvious - it requires the referred variable to already be known to the parser. Hence the following, which perhaps should be identical, will cause an error.

def traverse_tree_make_sizes_dct( v_dct_size_lst_pfn, v_dct_folderselfs, p_tree_path, p_multi_log):
    def traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_pfn_stock, p_tree_path, p_tree_depth, p_multi_log):
        nonlocal i_ncalls
        i_ncalls += 1
        print( "Folders traversed: " + str( i_ncalls) )
        return 
    i_ncalls = 0
    traverse_tree_recurse( v_dct_size_lst_pfn, v_dct_folderselfs, p_tree_path, 0, p_multi_log)
    return

End Note

This was written as notes for making a video. If that happens then a link will be edited into here.

Yet Another Functions of Functions Python Tutorial

techocodger — Mon, 13 Mar 2023 06:31:00 +0000

The Topic

Are you confused by Python functions that return functions and the places and ways that Python requires you to understand that? If so, perhaps this description of one example use case may help you.

Rationale

This article was written this as a note-to-self - and if anyone finds it helpful then that's good too. Some opinions will be expressed that hopefully are relevant to the code that's being discussed.

The Example

This example comes from a very small part of the process of writing the Python program Foldatry. However, that program is merely incidental for this article. It is however, a real example, from a real application that is being written and used.

Dictionary of Single Value Items

There is a part of Foldatry where it traverses a folder tree and noting the file extensions - the dot-something at the end of the name - as it goes.

To store those, it created a dictionary, where each new extension became a new key, and the value at the key was the count of files it had seen with that extension.

Thus, after the traversal was done, and the dictionary built, it was desired to output the findings, in order from the extension with the most files found, to the least found.

Something to note about Python and dictionaries, is that its idea of how the keys are has changed. In earlier Python the .keys() list was either considered unsorted or would be in the order that the keys were added. Perhaps to remove the ambiguity and/or to match the under-the-hood implementation of Python, that list is now defined as being tp preserve their order of insertion.

So, with the dictionary collected and held as the variable i_dct_counts - if we didn't care at all about the order to print them, we could do:

for k_ext in i_dct_counts.keys() :
    print( i_dct_counts[ k_ext ] )

for the Python pedants, yes, this is overlooking that iterating over the dictionary inherently goes through the keys - without explictly calling on .keys()

Most likely though, what a user wants to see, is which extension accounts for the most files, so to do that we will want to re-sort the list of keys - to be in descending order by the discovered counts.

To do that, we can use a generic sorted function that Python offers. The code will then look like:

lst_keys_sorted = sorted( i_dct_counts, key=i_dct_counts.get) 
for k_ext in lst_keys_sorted :
    print( i_dct_counts[ k_ext ] )

Frankly that kind of thing - the use of sorted is easy to look up online and plug into place - and indeed that's what was done here. In doing so, note that the sorted function needed to be passed the slighlty non-obvious thing: i_dct_counts.get but it clearly worked so all was good.

Dictionary of Tuples

For a revision of the program, it was decided to have the dictionary also hold information about the sizes of the files it found. To that end a "named tuple" was made to be used as the new data item.

Here's the definition for it:

tpl_Extn_Info = namedtuple('tpl_Extn_Info', 'ei_Count ei_SizeSum ei_SizeMin ei_SizeMax')

for which you can ignore the Pythonist mechanics and just see it as a non-simple data type to hold named elements of:

ei_Count - to hold the counts of files
ei_SizeSum - to hold the total size of files encountered
ei_SizeMin - to hold the largest size of file encountered
ei_SizeMax - to hold the smallest size of file encountered

where these are all things that could just be updated as files are encountered during the traversal, and are meaningful at the end of the traversal.

This change required some other bits of code to handle creating and updating these tuples during the folder tree travervals - but those don't matter for this tutorial, where we will merely assume it resuls in a suitably created dictionary of tuples.

But what this did change is what happens where this line executes:

lst_keys_sorted = sorted( i_dct_counts, key=i_dct_counts.get)

Because unlike before, the part key=i_dct_counts.get no longer tells the sorted function how to get a simple value that can control the sorting - and instead that will now deliver a tuple. This causes an error at run time, from inside the sorted function - because it "gets" something that has no defined comparison operation.

This means needing to have to done one of two things:

have our dictionary item be a kind of thing that is inherently sortable ;
provide a better function to sorted than the .get

Here we will ignore the first of those options - partly because it just wasn't the implemented resolution, but also because requires tackling a different aspect of how Python operates - and would thus be a tutorial about something else.

So the question then, became: what kind of function needs to be provided to sorted ? And how do we make such a thing?

Function that returns a Function

It is very tempting to show the blundering steps that was taken to work this out. It didn't take long - maybe 20 or 30 minutes - but those steps are harder to write about.

And to be very clear, this is certainly not claiming this is the best way to do this.

So let's jump to the code, and then talk it through.

tpl_Extn_Info = namedtuple('tpl_Extn_Info', 'ei_Count ei_SizeSum ei_SizeMin ei_SizeMax')

def ei_Count_of( p_tpl_Extn_Info ):
    return p_tpl_Extn_Info.ei_Count

def ei_Count_at_key( p_dct_tpl_Extn_Info, p_key ):
    return ei_Count_of( p_dct_tpl_Extn_Info[ p_key ] )

def fn_ei_Count_at_key( p_dct_tpl_Extn_Info ):
    def fn_ei_Count_of_dct_at_key( p_key ):
        return ei_Count_at_key( p_dct_tpl_Extn_Info, p_key )
    return fn_ei_Count_of_dct_at_key

The first function is simple - ei_Count_of when passed a tuple, will return its ei_Count value. Of course, when you have a particular tuple item the code for this is so trivial as to not be worth writing a function - but we know our goal here is to have a function we can quote. At first glance, this can seem to be the function required by sorted but cannot work because there is no way to tell sorted about this function as operating on the specific dictionary that it is dealing with.

The next function ei_Count_at_key is one that can be passed two things, the dictionary and the key, and will then return the count for that key. At first glance, this can seem to be the function required by sorted but cannot work because sorted needs to given a function that only takes the key as a parameter.

Finally we have the tricky bit. This is a function with a nested function. While that idea is not itself obscure, the reason for doing this is different to the reason we often do this kind of nesting - because here the reason for the inner function is so that it rather than what it does at execution time, is what the outer function returns.

Because, yes, when we call fn_ei_Count_at_key - passing it the dictionary we have in mind - what we get back is a function - and notably is a function that is customised for just that dictionary.

The reason this works is perhaps subtle - it is because the inner function makes use of the parameter passed to the outer function, so that it (the inner function) does not need the dictionary as a parameter. This makes the inner function - customised by the outer call - to be the kind of function that sorted needs to be told to use.

And here is how that new function gets used for supplying to the sorted function.

lst_keys_sorted = sorted( i_dct_counts, key=fn_ei_Count_at_key( i_dct_counts), reverse=True )
for k_ext in lst_keys_sorted :
    print( i_dct_counts[ k_ext ] )

A major part of understanding what happens at run-time is that when the above line is executed, the key= calling clause derives a function for the specific dictionary and then passed that in to the sorted function for it to use as it iterates through the dictionary.

A lesser part to be clear about there, is that when it does that iteration, it will iterate through the keys of the dictionary but operate on the items of the dictionary. Depending on your viewpoint, that is either obvious, or a subtle thing about handling dictionaries. It is worth noting that sorted is quite generic, and operates on anything (or almost anything?) that is iterable.

Process of Discovery and Aftermath Options

A reality of those functions, is that they represent the steps of building a way to having a function to pass to the sorted function. What is needed is a function, not to execute immediately in the sorted call, but for it to use as it iterates through the dictionary. Hence started by writing functions - rather than object de-references.

Indeed the meta-function approach arrived from realising that sorted was not happy to be given the function ei_Count_at_key or even a use of ei_Count_of combined with the get from the simpler non-tuple method.

Having solved the problem - of how to get a suitably sorted list of keys - the functions have been left in place. But should they now be revised to be more Pythonic, perhaps even to use a lambda in the sorted line?

Perhaps as written makes it quite clear what is going on. In terms of performance, there isn't much concern because the set of extensions is generally small - often finding between 5 and 100 discovered extension in typical usage.

Also, consider that maybe:

code that is more brief can be problematic by depending on deeper understanding of Python at run-time;
there isn't a problem with functions that are constructed but only used once - i.e. that this is not a good enough reason to use nameless methods, such as lambda

Walkthrough Video 006 - The Process Of Doing Tedious Code Changes

techocodger — Fri, 30 Dec 2022 14:57:37 +0000

Let's be blunt: sometimes, coding is just tedious. It is tedious and requires an unglamorous commitment to working through something consistently.

The classic situation for this, is where you want to achieve some kind of uniformity so as to make later changes less bothersome. Even though, there's some chance that you will never end up making another change. That's merely one of the paradoxes of good coding - or at least, what I consider "good coding" to mean.

This is the companion post for the video: TechoCodger - Walkthrough Video 006

The Example

I wasn't happy with how fiddly a particular pattern inside my program was proving to be. This revolved around the instances of a combination of:

a drop-down aka pull-down menu control
the values displayed on that control versus the internal representation of the value.

I had an idea that there was a better way to integrate those two things so that they both referenced the same internal concept. I did some experiments and found a solution that I liked - but it would require rewriting code every single place where the original mixed-concept was used so that it could be replaced by using the new concept.

But, this "internal concept" is itself only an archetype - throughout the program there were multiple similar combinations that I wanted to change over from the old method to the new style.

The purpose of this article is to walk through the example of making that change for the initial instance - and in such a way that we're ready to do the same for the other instances. It is about accepting that it will be tedious, and finding a good-enough way to work through the whole process.

As an extra requirement, I want:

to always be able to run the current code;
to be able to quickly flick a switch between using the old code and the new;
to keep all the multiple "flick a switch" options viable until the very last instance has been achieved.

The reasoning behind that last point is that my initial "new method" was based on making something "good enough" for the first instance. I won't really know it is good enough for all of the instances until the last has been done.

What that means in Python code

The example comes from work on the program Foldatry.

btw this is not a veiled promotion for that program, which I only consider to be a prototype, something written for my own use, and gets used as an example simply because the idea for this article came during work on its code.

So, for this program I'm writing, I recently made two coincident changes:

changed the definition and features for an Enumeration type (in the module "commontry")
made a reworked "Combobox" control by making a new class based on the Tkinter Combobox control.

With those as available new features, I then wrote and proved successful the usage of them.

For this walk-through, we'll take it as read that there is an Enumeration defined as:

@unique
class DeletionMode(Enum):
    DryRun = "DelMod_Dry"
    DeleteNow = "DelMod_Delete"
    Bash = "DelMod_Bash"

The enumeration is supported by some additional functions, but we'll let those be mentioned in context as they arise.

What I will show here is how I implemented the first instance, but with a mind to being a process I knew I would have to repeat for similar adaptations.

Preparing

As the basis for the change will include
I could search for DeletionMode_GetTuple

    t01_str_prunatry_mode = m_tkntr.StringVar() 
    t01_cb_prunatry_modes = m_tkntr_ttk.Combobox( t01_frame11, state="readonly", width = 20, textvariable = t01_str_prunatry_mode) 
    t01_cb_prunatry_modes['values'] = cmntry.DeletionMode_GetTuple()
    t01_str_prunatry_mode.set( cmntry.DeletionMode_Default_Value() )
    t01_cb_prunatry_modes.pack( side = m_tkntr.LEFT, padx=tk_padx(), pady=tk_pady() )

The code that will replace this, will be to create a Combobox object from the new type PairTupleCombobox with its associated calls to new features for the Enumeration class ( DeletionMode_LstPairTpls and DeletionMode_Default )

i.e.

    t01_cb_prunatry_mode = PairTupleCombobox( t01_frame11, cmntry.DeletionMode_LstPairTpls(), cmntry.DeletionMode_Default(), state="readonly", width = 50) 
    t01_cb_prunatry_mode.pack( side = m_tkntr.LEFT, padx=tk_padx(), pady=tk_pady() )

But, as an at-home amateur coder, I want my code to keep working as is until I'm ready to throw a switch and then test the replacements.

So what I do is:

leave the old aspects of the Enumeration intact and merely add the new support functions
bracket the changes I make in Foldatry with boolean flags.

    t_t01_old_combo_DeletionMode = True
    if t_t01_old_combo_DeletionMode :
        # the old code
    else :
        # the new code

Now, one detail to note here. In most other languages I would add my boolean control flag: t_t01_old_combo_DeletionMode at a global level. That way when it comes time to "throw the swtich" I just change a single True to a False. For reasons I won't go into, I don't trust the scoping of Python to be clear enough that I want to do that here. ([^1])

So what I will do - seeing as these control switches are all going to be temporary - is to always have the line pair:

    t_t01_old_combo_DeletionMode = True
    if t_t01_old_combo_DeletionMode :

in every single place where I'm making these changes. In practice it will merely mean that I will perform the changeover - from True to False - using a "replace all" in my editor instead of editing a single line. It's not a big deal.

First Phase - Locate and Insert If Blocks

The first time through, what I am doing is finding all the places where there will need to be a controllable switch between old and new methods.

For each I will insert the line pair before the existing code and put an empty else : clause after the existing code. Here is the first case:

            t_t01_old_combo_DeletionMode = True
            if t_t01_old_combo_DeletionMode :
                setting_value = cmntry.DeletionMode_EnumOfString( t01_str_prunatry_mode.get() )
            else :
                pass

The orginal was just the line: setting_value = cmntry.DeletionMode_EnumOfString( t01_str_prunatry_mode.get() )

Note that Python syntax requires a pass otherwise the else will cause an error.

Here is the next one:

            t_t01_old_combo_DeletionMode = True
            if t_t01_old_combo_DeletionMode :
                t01_str_prunatry_mode.set( cmntry.DeletionMode_EnumValueString( val ) )
            else :
                pass

We can repeat these edits, mainly ignoring what the line actually does or needs.

                t_t01_old_combo_DeletionMode = True
                if t_t01_old_combo_DeletionMode :
                    t_prunatry_mode_show = t01_str_prunatry_mode.get()
                else :
                    pass

Another,

            t_t01_old_combo_DeletionMode = True
            if t_t01_old_combo_DeletionMode :
                t_prunatry_mode_ok = t01_str_prunatry_mode.get() in cmntry.DeletionMode_GetList()
            else :
                pass

and another - we will come back to what each these really does, when we come around again to write the replacements.

                t_t01_old_combo_DeletionMode = True
                if t_t01_old_combo_DeletionMode :
                    i_enum_prunatry_mode = cmntry.DeletionMode_EnumOfString( t01_str_prunatry_mode.get() )
                else :
                    pass

Finally - due to the way I like to structure my Tkinter code, the actual creation of the Tkinger GUI objects comes last - so here we see the creation of the Combobox GUI control.

    t_t01_old_combo_DeletionMode = True
    if t_t01_old_combo_DeletionMode :
        t01_str_prunatry_mode = m_tkntr.StringVar() 
        t01_cb_prunatry_modes = m_tkntr_ttk.Combobox( t01_frame11, state="readonly", width = 20, textvariable = t01_str_prunatry_mode) 
        t01_cb_prunatry_modes['values'] = cmntry.DeletionMode_GetTuple()
        t01_str_prunatry_mode.set( cmntry.DeletionMode_Default_Value() )
        t01_cb_prunatry_modes.pack( side = m_tkntr.LEFT, padx=tk_padx(), pady=tk_pady() )
    else :
        pass

To state the obvious, all of those if statements are set to evaluate as True and thereby the code at run time remains unaffected.

Second Phase - Write In Else Clauses

Having identified all the places where all the replacements will need to be inserted, we can now work through the process of writing them all.

Here is the first one. This was about reading the state of the Combobox for then saving settings into a configuration file.

            t_t01_old_combo_DeletionMode = False
            if t_t01_old_combo_DeletionMode :
                setting_value = cmntry.DeletionMode_EnumOfString( t01_str_prunatry_mode.get() )
            else :
                setting_value = t01_cb_prunatry_mode.getSelectedKey()

The second. This was about using settings that had been loaded from a configuration file to reset the Combobox.

            t_t01_old_combo_DeletionMode = False
            if t_t01_old_combo_DeletionMode :
                t01_str_prunatry_mode.set( cmntry.DeletionMode_EnumValueString( val ) )
            else :
                t01_cb_prunatry_mode.setSelectedKey( val )

The third - more about this later. This was just to have a string to display in the status window and logs.

                t_t01_old_combo_DeletionMode = False
                if t_t01_old_combo_DeletionMode :
                    t_prunatry_mode_show = t01_str_prunatry_mode.get()
                else :
                    t_prunatry_mode_show = cmntry.DeletionMode_String( t01_cb_prunatry_mode.getSelectedKey() )

The fourth. This was to feed a Boolean check of whether an acceptable value had been set in the Combobox. As it happens, the change to the new type of Combobox means that this check is unnecessary, but for now we'll not tackle removing this part.

            t_t01_old_combo_DeletionMode = False
            if t_t01_old_combo_DeletionMode :
                t_prunatry_mode_ok = t01_str_prunatry_mode.get() in cmntry.DeletionMode_GetList()
            else :
                t_prunatry_mode_ok = t01_cb_prunatry_mode.getSelectedKey() in cmntry.DeletionMode

The fifth. This is where the actual mode value - as an Enumeration - has to be pulled from the Combobox. In the old method I was getting a string from the Combobox and then having to call a function to get the matching Enumeration. In the replacement I directly get an Enumeration from the new class object.

                t_t01_old_combo_DeletionMode = False
                if t_t01_old_combo_DeletionMode :
                    i_enum_prunatry_mode = cmntry.DeletionMode_EnumOfString( t01_str_prunatry_mode.get() )
                else :
                    i_enum_prunatry_mode = t01_cb_prunatry_mode.getSelectedKey()

Finally, here is where the actual Combobox control gets created.

    t_t01_old_combo_DeletionMode = False
    if t_t01_old_combo_DeletionMode :
        t01_str_prunatry_mode = m_tkntr.StringVar() 
        t01_cb_prunatry_modes = m_tkntr_ttk.Combobox( t01_frame11, state="readonly", width = 20, textvariable = t01_str_prunatry_mode) 
        t01_cb_prunatry_modes['values'] = cmntry.DeletionMode_GetTuple()
        t01_str_prunatry_mode.set( cmntry.DeletionMode_Default_Value() )
        t01_cb_prunatry_modes.pack( side = m_tkntr.LEFT, padx=tk_padx(), pady=tk_pady() )
    else :
        t01_cb_prunatry_mode = PairTupleCombobox( t01_frame11, cmntry.DeletionMode_LstPairTpls(), cmntry.DeletionMode_Default(), state="readonly", width = 50) 
        t01_cb_prunatry_mode.pack( side = m_tkntr.LEFT, padx=tk_padx(), pady=tk_pady() )

Complexities

Some those changes were slightly more complex than they might seem.

For example, in the third example above, the original was a one-line with a call to fetch the visible value of the Combobox, which ironically is no longer a valid thing to do with the new Combox

t02_label_paths_process_status.configure( text = "Abandoned processing in mode:" + t01_str_prunatry_mode.get())

In which case, one apparent way to tackle this is to replicate the configuring of the label for True and False - i.e.

t_t01_old_combo_DeletionMode = False
if t_t01_old_combo_DeletionMode :
    t02_label_paths_process_status.configure( text = "Abandoned processing in mode:" + t01_str_prunatry_mode.get())
else :
    t02_label_paths_process_status.configure( text = "Abandoned processing in mode:" + cmntry.DeletionMode_String( t01_cb_prunatry_mode.getSelectedKey() )

But I didn't like the feel of doing that, so I reworked it to set an interim variable as the thing governed by the if and then have the label configuration appear only once.

Hence, first doing this:

t_t01_old_combo_DeletionMode = False
if t_t01_old_combo_DeletionMode :
    t_prunatry_mode_show = t01_str_prunatry_mode.get()
else :
    t_prunatry_mode_show = cmntry.DeletionMode_String( t01_cb_prunatry_mode.getSelectedKey() )

and then doing this:

t02_label_paths_process_status.configure( text = "Abandoned processing in mode:" + t_prunatry_mode_show )

Now you might (rightly) ask: why bother with such sophistries when this is all code that will eventually be deleted after all the changes prove themselves successful?

To me, this is about establishing and holding various "good routine practices". And the answer to the "why?" question is that I'm going to have quite a few of these sequences of changes to work through. I'm therefore interested in using a method where there is less that can go wrong.

This is one of many coding paradoxes - extra work to reduce risk, or to reduce future extra work - and I assume that all coders go about these in ways that suit themselves. I can tell you what I do and why I do it, but not whether you should also do it.

Footnotes:

([^1]: It is perhaps more accurate to say I don't trust my clarity about the scoping rationale in Python. This being a problem of writing in multiple languages.

Coding Walkthrough 004 - Supplanting Functions

techocodger — Tue, 06 Dec 2022 11:53:16 +0000

These notes are a companion piece for the video:

Coding Walkthrough 004 - Supplanting Functions

Topic: Supplanting Functions

These are the show notes for a video demonstration of changing an existing Python program to add new functions that will supplant some existing function calls.

The purpose of the video is to show an example of doing this in the context of a real and non-trivial program. In this case it happens during an edit of "Foldatry" which is a multi-purpose file and folder tool that I am writing in Python.

Note: all the code on this page is actually "air code", written in a mere text editor as planned code which is then clipped in during the video. I've deliberately left intact the errors that quickly became apparent as development continued.

The Example Situation

From some runs of Foldatry I had the wish that I could know what quantity of data storage was at issue. While there are several places where I'd want to make that change, for this run-through I'll just look at one of its parts - Congruentry.

Note: if you want to inspect the code (as it was when these notes were written), use this specific link:

congruentry.py as at 15 Nov 2022

The Congruentry module is all about comparing two folder locations and determining whether they and all their sub-folders are the same. The main use for this is for confirming that a copy process - done by some other tool - was completed perfectly. While the simplest answer to that is either "Yes, the same" or "No, not the same" in practice when the answer is "No" we want to know something useful about that.

Congruentry is also capable of indicating when one of the two folders is a full subset of the other, in which case it can assemble a list of the additional items in the superset folder.

But, it does not bring back any information about how much file size those differences amount to. One reason that currently does not happen is because it uses three functions from the stock filecmp library to enact the comparisons - and none of those return any information about file sizes.

As it happens, for other reasons, I am also writing my own replacement for the stock filecmp library, as another module inside Foldatry - but it is not ready yet.

So our target here is that we want to replace use of the stock filecmp library with calls to a "mock" set of functions, so that:

for now they will just call the filecmp library anyway;
then later they can be changed to call the new module when it has been written (and proven).

Actually we can go one better than that plan, and add an extra change in the sequence:

initially they will just call the filecmp library anyway;
then extensions can be made, so that are the filecmp cals, the list of differences can be traversed to collect file size information;
then later again they can be changed to call the new module when it has been written (and proven).

What To Replace

Ok, so where are the calls that we will supplant?

We can easily trace them, because the top of the module congruentry.py has this import:

import filecmp as im_filecmp

Due to the alias there, the calls will all be like im_filecmp.something

And here they are (also showing the function nesting of their locations);

first, we have a call that does most of the work:

def congruency_check_trees( cct_p_Path_A, cct_p_Path_B, cct_p_Stringent, cct_p_badwinfname, cct_p_badwinchars, \

    def trees_pass( tp_p_Stringent ):

        def congruency_check_subtrees( ccs_p_Path_A, ccs_p_Path_B, ccs_p_depth, ccs_p_still_congruent, ccs_p_extra_side ):

            dcmp = im_filecmp.dircmp( ccs_p_Path_A, ccs_p_Path_B ) # ? shallow=True

wherein the function called was dircmp which returned an object dcmp that gets further interactions afterwards - we'll discuss the details of that later.

second, we have a call that is used to perform the "stringent" file comparisons of file contents:

    def congruency_check_subtrees( ccs_p_Path_A, ccs_p_Path_B, ccs_p_depth, ccs_p_still_congruent, ccs_p_extra_side ):

            (fils_match, fils_mismatch, fils_errors) = im_filecmp.cmpfiles(
                ccs_p_Path_A, ccs_p_Path_B, dcmp.common_files, shallow=False)

wherein the function called was cmpfiles which returned three lists of filenames.

third, we have a call that just compares two specific files:

        def congruency_check_subtrees( ccs_p_Path_A, ccs_p_Path_B, ccs_p_depth, ccs_p_still_congruent, ccs_p_extra_side ):

                        chckd_XAB_congruent = im_filecmp.cmp( pathfile_a, pathfile_b, shallow=False)

wherein the function called was cmp which returned a boolean.

Note: that part gets done as a followup comparison that copes with filenames that are non-identical - e.g. because of different character encodings in the file systems. For this excercise we can ignore the why of all that.

fourth, because the Congruentry module provides a direct file vs file comparison, we have another call that just compares two specific files:

def congruentry_files_command( pathfile_a, pathfile_b, p_multi_log):

    chckd_congruent = im_filecmp.cmp(pathfile_a, pathfile_b, shallow=False)

wherein the function called was again cmp which returned a boolean.

So, that was usage of:

dircmp = compare directories, returning an object - note that some comparison actions don't happen until parts of the object are called
cmpfiles = compare the files in two directories
cmp = compare two specified files

which are therefore the features we need to make supplanting functions. We may as well do these inside the Congruentry module.

The Mocking

Mock function names

We'll need three new functions, Here is my plan:

fcmp_for_two_dirs_compare_get_object to replace dircmp
fcmp_for_dirs_compare_files_get_lists = to replace cmpfiles
fcmp_for_two_files_compare_contents_get_bool = to replace cmp

Now it should be said, that it took a few rounds of thought to get those names. Quite a bit of the following was written with the names being quite different. Eventually, I settled on a prefix fcmp_ for them all and the names to be a sequence of "for this" then "do this" then "get this".

Naive mock up

Time to use the names and start framing them into being Python functions.

def fcmp_for_two_dirs_compare_get_object()
def fcmp_for_dirs_compare_files_get_lists()
def fcmp_for_two_files_compare_contents_get_bool( )

Parameters

Now let's add the the parameters. As my first target is to simply pass-through to the filecmp library, I'll set out the same parameter sets.

def fcmp_for_two_dirs_compare_get_object( p_path_a, p_path_b, p_shallow=True )
def fcmp_for_dirs_compare_files_get_lists( p_path_a, p_path_b, p_lst_common_files, p_shallow=False )
def fcmp_for_two_files_compare_contents_get_bool( p_pathfile_a, p_pathfile_b, p_shallow=False )

I am cheating however, in only bothering about the parameters I'm currently using in my own code.

Usable Mocks

Let's now add enough code to pass through to the existing calls. I don't even need to bother with local variables and instead just put the calls in the return lines.

def fcmp_for_two_dirs_compare_get_object( p_path_a, p_path_b, p_shallow=True)
    return im_filecmp.dircmp( p_path_a, p_path_b, p_shallow )

def fcmp_for_dirs_compare_files_get_lists( p_path_a, p_path_b, p_lst_common_files, p_shallow=False )
    return im_filecmp.cmpfiles( p_path_a, p_path_b, p_lst_common_files, p_shallow)

def fcmp_for_two_files_compare_contents_get_bool( p_pathfile_a, p_pathfile_b, p_shallow=False )
    return im_filecmp.cmp(pathfile_a, pathfile_b, shallow=False)

Customised Round One

But, the whole point of this exercise was to enable some changes, so let's do the first parts of how that might work.

For the simple comparison of two files (fcmp_for_two_files_compare_contents_get_bool), let's make it return whether or not a deep/content comparison was required and done.

For the comparison of just the files in the two directories (fcmp_for_dirs_compare_files_get_lists), let's return the total file size of those. For this, we'll put two extra functions inside it (pathfile_filesize and pathfile_filesizes_sum). And, one of those will, for now, only be a mock function as it will just always return None - we can work out what the valid Python for doing that is later.

def fcmp_for_two_dirs_compare_get_object( p_path_a, p_path_b, p_shallow=True):
    r_dcmp = im_filecmp.dircmp( p_path_a, p_path_b, p_shallow )
    return r_dcmp

def fcmp_for_dirs_compare_files_get_lists( p_path_a, p_path_b, p_lst_common_files, p_shallow=False ):
    def pathfile_filesize( p_path, p_file):
        return None
    def pathfile_filesizes_sum( p_path, p_lst_files):
        klang = False
        r_sum = 0
        for i_file in p_lst_files :
            i_sum = pathfile_filesize( p_path, i_file)
            if not i_sum is None :
                r_sum = r_sum + i_sum
            else:
                # a single failed filesize means the sum is invalid
                klang = klang or True
        if klang :
                r_sum = -1
        return r_sum
    (r_files_match, r_files_mismatch, r_files_errors) = im_filecmp.cmpfiles( p_path_a, p_path_b, p_lst_common_files, p_shallow)
    r_match_size_sum = pathfile_filesizes_sum( p_path_a, r_files_match)
    return r_files_match, r_files_mismatch, r_files_errors, r_match_size_sum

def fcmp_for_two_files_compare_contents_get_bool( p_pathfile_a, p_pathfile_b, p_shallow=False ):
    r_same_shallow = im_filecmp.cmp(pathfile_a, pathfile_b, True)
    if r_same_shallow and p_shallow :
        r_same_content = im_filecmp.cmp(pathfile_a, pathfile_b, False)
    else:
        r_same_content = False
    return r_same_shallow, r_same_content

Customised Round Two

In which we extend the functionality of the object method so that it can return the collective sum of the files found to be in common.

As part of this we can lean on the two functions we've already built inside the function fcmp_for_dirs_compare_files_get_lists but to have those available to both that and the "object" function, we'll bring them to the outside. For clarity we'll add our prefix fcmp_ to their names.

# support functions

def fcmp_pathfile_filesize( p_path, p_file):
    return None

def fcmp_pathfile_filesizes_sum( p_path, p_lst_files):
    klang = False
    r_sum = 0
    for i_file in p_lst_files :
        i_sum = fcmp_pathfile_filesize( p_path, i_file)
        if not i_sum is None :
            r_sum = r_sum + i_sum
        else:
            # a single failed filesize means the sum is invalid
            klang = klang or True
    if klang :
            r_sum = -1
    return r_sum

# the replacement functions

def fcmp_for_two_dirs_compare_get_object( p_path_a, p_path_b, p_shallow=True):
    r_dcmp = im_filecmp.dircmp( p_path_a, p_path_b, p_shallow )
    i_files_match = r_dcmp.same_files
    r_match_size_sum = fcmp_pathfile_filesizes_sum( p_path_a, i_files_match)
    return r_dcmp, r_match_size_sum

def fcmp_for_dirs_compare_files_get_lists( p_path_a, p_path_b, p_lst_common_files, p_shallow=False ):
    (r_files_match, r_files_mismatch, r_files_errors) = im_filecmp.cmpfiles( p_path_a, p_path_b, p_lst_common_files, p_shallow)
    r_match_size_sum = fcmp_pathfile_filesizes_sum( p_path_a, r_files_match)
    return r_files_match, r_files_mismatch, r_files_errors, r_match_size_sum

def fcmp_for_two_files_compare_contents_get_bool( p_pathfile_a, p_pathfile_b, p_shallow=False ):
    r_same_shallow = im_filecmp.cmp(pathfile_a, pathfile_b, True)
    if r_same_shallow and p_shallow :
        r_same_content = im_filecmp.cmp(pathfile_a, pathfile_b, False)
    else:
        r_same_content = False
    return r_same_shallow, r_same_content

Ready for Implanting

The above has been a "paper exercise" - just written in my programming notes tool rather than in the IDE where I actually work on my Python program. So from here, the exercise will shift into one of pasting this code into there and seeing if it will actually work.

Enacting the Size

So far, we've left the function for getting the file sizes quite unable to actually do that.

def fcmp_pathfile_filesize( p_path, p_file):
    return None

As it happens, I already have ome code for doing this, as it was needed in another module.

In the Matchsubtry module we have aline:

    size = im_os.path.getsize(fpath)

A quick check confirms that Congruentry has the same library imported:

import os as im_os

As our intended function fcmp_pathfile_filesize is currently taking two parameters: p_path and p_file we will need something to combine them.

A quick read find the place in Congruentry where I already do that:

                    new_subpath_a = im_osp.join(ccs_p_Path_A, subdir)

So this give me enough to construct the pieces to make the function operative.

def fcmp_pathfile_filesize( p_path, p_file):
    i_pathfile = im_osp.join( p_path, p_file)
    r_size = im_os.path.getsize( i_pathfile)
    return r_size

Note that I didn't go straight to putting that all in the return line. I like giving myself the option of putitng in some print statements (or some other "debug" calls) as I go to implement this for the first time.
e.g.

def fcmp_pathfile_filesize( p_path, p_file):
    print( p_path, p_file)
    i_pathfile = im_osp.join( p_path, p_file)
    print( i_pathfile)
    r_size = im_os.path.getsize( i_pathfile)
    print( r_size)
    return r_size

As we're dealing with the file system here, it is wise to not assume that we won't have things going wrong at run time.

The simplest thing is to wrap each file system interaction with a try except pair.

def fcmp_pathfile_filesize( p_path, p_file):
    try:
        i_pathfile = im_osp.join( p_path, p_file)
        path_ok = True
    except:
        path_ok = False
    try:
        r_size = im_os.path.getsize( i_pathfile)
        size_ok = True
    except:
        size_ok = False
    if not size_ok :
        r_size = None
    return r_size

That's actually slightly ambiguous, as it attempts to call getsize regardless of whether the path+file construction gave an error. I like to have the possible errors kept separate.

def fcmp_pathfile_filesize( p_path, p_file):
    # first form the path
    try:
        i_pathfile = im_osp.join( p_path, p_file)
        path_ok = True
    except:
        path_ok = False
    # then get the size
    size_ok = False
    if path_ok :
        try:
            r_size = im_os.path.getsize( i_pathfile)
            size_ok = True
        except:
            pass
    if not size_ok :
        r_size = None
    return r_size

Note that there are many ways to code that logic - for various degrees of optimisation and/or being Pythonic. For now, just getting the logic valid and bulletproof will do.

Two Techocodger Videos

techocodger — Sun, 21 Aug 2022 14:23:42 +0000

As presaged in my hello post corresponding to this account, there is a YouTube channel.

Techocodger YouTube Channel

The channel was set up to host two videos I made as a pair of "coding walkthroughs".

These appear as the playlist:

Coding Walkthroughs

and the two videos are:

Coding Walkthrough 001 - Code Inspection
Coding Walkthrough 002 - Code Cascade

Here is a brief overview of them, there will be a fuller post as a companion to each one.

These coding walkthroughs are aimed at programmers who have been learning to code but perhaps haven't yet written much beyond the typical small coding examples.

I wanted to show what it is like dealing with a moderate amount of your own code, especially when you only work on it occasionally. The topic here is not really about the code itself, but rather the ordinary nature of coding.

These videos are meant to be like looking over the shoulder of a colleague as they code, from which you can politely walk away with your own thoughts of how you would do things better.

To that end, these are deliberately single-take recordings, with no edits. The indecision and rediscovery that you can watch me do, is largely what I want to demonstrate.

Note that this does mean that the commentary is an important part, so if you watch without sound you will only see some unimpressive meandering around some equally unimpressive code. At the time of writing I haven't seen how well the automated captioning on YouTube will turn out.

Coding Walkthrough 001 - Code Inspection

This walkthrough tackles revisiting the source code of a program so as to work out which parts will need to be modified for adding a desired new feature.

Some GUI controls have already been added and the intention is to rediscover into what parts of the program's structure those values will need to be passed.

This first session, was just about looking around, no code editing happens.

Coding Walkthrough 001 - Code Inspection

Coding Walkthrough 002 - Code Cascade

This walkthrough follows on from the previous video. It tackles actually adding code to pass the value of a new checkbox in the GUI down into the places where new code will need to be written.

The session shows just one of the many ways to tackle making a series of edits that will end up providing a new propagation of values through multiple layers of the program.

Coding Walkthrough 002 - Code Cascade

I do have longer companion articles (mostly) written for each of the videos and will probably publish those in due course.

Hello from a Techocodger

techocodger — Sun, 21 Aug 2022 14:20:00 +0000

Hi, I'm Techocodger and I'll be using this account to share some demonstrations of the dull and unglamorous side of coding. So if you want to be hip and cool and trendy about programming this probably is not for you.

I've been a self-taught, mediocre, sporadic and amateur grade programmer for decades.

I have made a couple of videos because I wanted to let beginner programmers see that you don't have to be a coding wizard or expert in language X to create a program that can seem to be complex.

Now, to be a bit more candid, "Techocodger" is an alter-ego that is just for this purpose. There is no "business" to this, nothing being sold or promoted, no social graph plan etc. I just felt like doing it. If it helps you, good, and if not, I'll apologise for wasting your time - or even perhaps for showing you my bad habits and techniques.

I do have a regular personal-style account here on Dev - this is really just a convenient separation of purpose. The connection is not anything secret and can probably be deduced from the video content.

DEV Community: techocodger

A Small Code Renaming Example in Python

Prefixes

The Outer Call

The Inner Call

Issues that Renaming Brings To Light

End Note

Yet Another Functions of Functions Python Tutorial

The Topic

Rationale

The Example

Dictionary of Single Value Items

Dictionary of Tuples

Function that returns a Function

Process of Discovery and Aftermath Options

Walkthrough Video 006 - The Process Of Doing Tedious Code Changes

The Example

What that means in Python code

Preparing

First Phase - Locate and Insert If Blocks

Second Phase - Write In Else Clauses

Complexities

Footnotes:

Coding Walkthrough 004 - Supplanting Functions

Topic: Supplanting Functions

The Example Situation

What To Replace

The Mocking

Mock function names

Naive mock up

Parameters

Usable Mocks

Customised Round One

Customised Round Two

Ready for Implanting

Enacting the Size

See Also

Two Techocodger Videos

Coding Walkthrough 001 - Code Inspection

Coding Walkthrough 002 - Code Cascade

Hello from a Techocodger