DEV Community: gallau

Lifecycle of a URL Request

gallau — Sun, 10 Sep 2023 22:34:35 +0000

Last week, I came across a valuable LinkedIn article by Brij Kishore Pandey detailing the journey of a "URL from Keyboard to Screen". I've written this blog to save its insights, but if you're in tech, I highly recommend reading it. The article breaks down the components and flow of a URL request.

The Journey of a URL: From Your Keyboard to a Fully Loaded Web Page! 🌐

Ever wondered what happens behind the scenes when you type a URL into your browser's address bar and hit Enter? 🤔

Let's demystify the magic that unfolds in milliseconds! ⏱️

🖥️ Client-Side Magic:

1️⃣ Initiating TCP Connection: Your browser initiates a TCP/IP 3-Way Handshake (SYN, SYN-ACK, ACK) to establish a connection.
2️⃣ HTTP Request: A GET request is sent to the server.
3️⃣ Browser Engine: The browser engine takes over, coordinating between the render engine and networking tasks.

🌐 The Role of DNS:

1️⃣ Root Domain Name Server: Helps in identifying the Top-Level Domain (TLD).
2️⃣ TLD Name Server: Points to the 2nd Level Domain Name Server.
3️⃣ 2nd & 3rd Level Domain Name Servers: They help in resolving the IP address of the website.

🖥️ Server-Side Magic:

1️⃣ Server Response: The server responds with status codes:

1xx: Information Message
2xx: Success
3xx: Redirects
4xx: Client Errors
5xx: Server Errors

🌍 Web Page Rendering:

1️⃣ Tokenizer & DOM Tree: HTML is parsed and a DOM tree is created.
2️⃣ CSSOM Tree: CSS is parsed into a CSS Object Model (CSSOM) tree.
3️⃣ Render Tree: DOM and CSSOM trees are combined.
4️⃣ Layout & Painting: The final layout is calculated and painted on the screen.

Your Web Page is Loaded!

Here's the image:

This post is inspired by and based on a sketch from @manekinekko on X(Formerly Twitter)
This Gif it licensed under : https://lnkd.in/eBAGdRtG

Performance Testing: An Incremental Approach

gallau — Sun, 09 Jul 2023 07:27:37 +0000

Introduction

Often, people from different technical and nontechnical backgrounds who are unfamiliar with performance testing principles approach me with a recurring problem. They seem to run the same test over and over again, hoping for different results each time. This ineffective and time-consuming approach stems from a lack of understanding of performance testing principles. Often they are new to the organisation are unfamiliar with the business and technical environment and are given a framework that has failed to idenitfy the cause of the issues which will assist stakeholders in resolving the underline issue.

In this article, I'll be outlining an approach that I have implemented in different contexts to resolve performance issues by providing a framework that assists in highlighting areas of concern.

Understanding the Purpose of Performance Testing

The first question to answer is, "Why are we doing performance testing?" Understanding the rationale behind the testing will guide the overall process and help in setting the right goals. Performance testing is not just about finding bugs or errors. It's also about ensuring that the software system will perform well under its expected workload. If, for example, the performance issue occurs during nightly batch processing of user transactions, and your test script only emulates users interacting with the system via a UI during the day, you'll need to realign your team's efforts to resolve the actual production issue.

Defining the ‘Done’

Before any testing is done, it is critical to have the definition of 'done' written down. This aligns with lean principles, where clear, actionable goals are vital. If it hasn't been articulated yet, engage the stakeholders to agree on an interim and overall goal. For instance, system stability could be a priority, with a target for a soak test to run overnight without any system failure. In lean thinking, this is akin to defining the target state or the desired end condition of a process. Defining 'done' in clear, measurable terms helps in minimizing waste (like unnecessary testing) and ensuring everyone is focused on achieving the same outcome.

Aligning Test Environment and Infrastructure

With a clearly defined and agreed-upon goal, the next step involves reviewing and outlining the requirements for the test environment, infrastructure, test scripts, and results.

Start by assessing the observability tools in your arsenal (e.g., CPU and memory monitors, HTTP access logs, error logs), and identify any additional tools that may be needed to enhance your insights into the test results.

Ensure to inquire whether the production monitoring setup is replicated on the test infrastructure. This is important because such monitoring can add to the system load and might, in rare instances, cause errors that could affect your testing and results.

Consider the presence of any software agents running on the production environment, such as antivirus, licensing agents, or auditing agents. These should also be set up and configured in the same way on your test environment since they contribute to the system load.

Furthermore, it is crucial to have alerts for low disk space. In the case of frequent test runs, disk space can be quickly consumed. Therefore, it is recommended to have clean-up scripts ready for clearing disk space and resetting databases between test runs.

Test Environment Setup

Your test environment should mimic your production environment as closely as possible. This is to ensure that your tests accurately reflect real-world conditions. For external dependencies, consider using a mock to exclude the action from the test and reduce noise. The lifespan of the environment before it needs to be recycled should dictate the length of any soak test.

Building a Production Load Model

It is critical to build a model of the production load. This involves identifying and replicating the most common transactions based on volume and system utilization in your test scripts. Log analysis tools like Splunk or Honeycomb are invaluable in helping to identify the most common transactions.

Engage with the operations team to gather data on past high-stress events, such as Cyber Monday sales. These events can provide insights into extreme system behaviours. Examining daily transaction volumes over an extended period will reveal patterns, helping to identify periods of high load. In collaboration with stakeholders, determine what constitutes a typical business day versus a high-volume day.

Once you have identified these patterns, analyze and categorize the transactions by volume, ordering them from highest to lowest by percentage. Compare this production load with your existing performance test load to determine if your test scenario accurately emulates production. Pay particular attention to the counts of errors or exceptions between the production and test environments. Differences here could highlight patches applied in production but missed in test environments, or variations in log levels, which could inadvertently add unnecessary load.

Focus on the transactions that generate approximately 80% of the system load. Consult with business stakeholders to determine if any less frequent transactions are mission-critical and should be included in the test script.

After you've built a production scenario model, the next step is to adapt your test model to emulate it. Present this proposed model to your stakeholders to gain their acceptance and agreement. This will help ensure that the model accurately represents the system's real-world use and meets stakeholder expectations. Once you have stakeholder approval, make sure you can execute the test model twice for the same duration to ensure results can be reproduced. This ensures that the model is robust and capable of providing reliable and consistent data for analysis."

Consider leveraging database snapshots to reset the test database between runs. This practice helps prevent the skewing of performance results due to accumulating data, such as adding multiple quotes to the same user. As an example, in one case, a user had millions of quotes associated with their profile, causing a report generation to take hours rather than seconds due to the large volume of data retrieved.

Aligning with Production Service Level Agreements

In performance testing, it is crucial to understand the Service Level Agreement (SLA) for production-level transaction response times. These SLAs act as the guiding benchmark, shaping the focus and expectations of your testing activities.

When defining the SLA in your test environment, it's paramount to align it with the production system's SLA. For instance, if responsiveness in the production environment is gauged at the 95th percentile, utilize the same percentile when interpreting and reporting your performance testing results. This alignment guarantees consistency in the measurement process and fosters a meaningful comparison between your test outcomes and the production SLAs.

In situations where established production SLAs are absent, it's advisable to negotiate interim SLAs based on current production data. This provisional benchmark not only offers a target to work towards but also serves as a monitoring tool. It can help identify if the production situation is worsening or if there has been a significant system change. These interim SLAs, thus, ensure the testing process remains purposeful and effective, even in the absence of established production benchmarks.

Developing and Debugging Test Scripts and Environment

After receiving stakeholder approval for the production scenario model, it's time to get down to the nitty-gritty of testing.

Start by running each script with a single user, twenty times. The objective here is to ensure the repeatability of results and eliminate any randomness in the script settings.

Check if the response time results comply with the production Service Level Agreement (SLA) at the agreed percentile (in our case, the 95th percentile). Most performance tools report on the minimum, average, percentile, and maximum response times. Consider the minimum value - this represents the fastest transaction in the test run. If this transaction fails to meet the SLA, bring this to the stakeholders' attention for investigation. Having a representative from the development and DevOps team at this stage can expedite troubleshooting.

When the script meets the SLA with a single user, ensure you can replicate these results twice in a row. Apply this process to each test script. You may need to run specific scripts to generate enough setup data for these tests to execute successfully. The primary aim here is to eliminate any low-hanging issues that could distort results at higher volumes.

Subsequently, adopt the user numbers as per the production model - for example, 25, 50, and 75 concurrent users. Run each script individually at these user volumes and verify if they align with the Production SLA. During the evaluation, pay particular attention to the standard deviation of response times for each transaction.

For instance, if you observe that a transaction, say "User Login", has an average response time of 2 seconds but a high standard deviation, such as 1.5 deviations, this could be a symptom of a bottleneck or inconsistent performance. The high standard deviation indicates that while some users are able to login quickly, others are experiencing significant delays. This inconsistency could be due to several factors such as inefficient database queries, issues with server resources, or a poorly optimized login service. It's crucial to delve into these fluctuations and investigate the root cause to ensure a consistent and smooth user experience.

After each run, scrutinize the logs for CPU utilization, memory usage, and error count. Confirm the environment monitoring and alerting configurations, and ensure you can reproduce test results at least twice. This comprehensive approach allows you to not only validate the performance under expected load but also uncover potential issues that may otherwise remain hidden under the hood.

These steps aim to identify functional problems with scripts or environment setup and remove "low hanging" issues before you run your combined tests. Interim tuning allows other technical stakeholders to test out their observability tools. Once all of the individual tests can be run at the agreed load levels in a repeatable way, then it's time for the combined tests.

This testing process will yield a wealth of data that can be graphed to show system performance at various load levels. This can help communicate the system's capabilities and limitations to a non-technical audience, thereby fostering understanding and consensus on performance objectives and system improvements.

Combining the Tests and Verifying System Performance

Once all the individual scripts pass the agreed load benchmarks, it's time to amalgamate the tests.

Draw from the business scenario ratios that were established earlier and execute the scripts with the agreed user numbers and volumes. Note that achieving an exact replication of numbers at high volumes can be challenging. To account for this, it's recommended to agree on a tolerance range with your stakeholders. For instance, if the target hourly volume is 100 transactions, you might set an acceptable range from 90 to 110 transactions.

Commence your combined testing at the lower end of your user load, for example, 25. Validate your metrics and confirm you can reproduce the test results at least twice. This duplication aids in ensuring the consistency of your results and reduces the chance of overlooking issues.

Should your tests fail at this initial user load, engage your stakeholders for debugging and remediation. Once resolved, validate your tests at the higher user loads (in this case, 50 and 75).

Graphing your results can be beneficial in visually demonstrating progress against the agreed targets to both technical and non-technical stakeholders.

When you reach this stage of performance testing, the issues you'll be addressing might be more challenging to resolve, but you'll be less distracted by low-level issues. It's also possible you might need to augment system resources to meet the performance expectations set by your stakeholders.

Reiterate the importance of being able to reproduce your test results at least twice. Consistent replication not only confirms the accuracy of your testing but also instills confidence in the test results amongst your stakeholders.

Conducting Additional Tests & Practices

At this stage, there are a few additional testing types that could offer value to your project. Discuss these options with your stakeholders. Given that you have all the resources aligned, these tests may provide crucial insights for future performance scenarios.

a. 72-Hour Soak Test
This test assesses the system's resilience over an extended period. It can help identify problems like slow memory leaks and other lingering issues. During this test, consider using a 'wave scenario.' This involves dividing each user group into two separate groups: one group with a 60-second wait time between iterations and another with a 120-second wait time. Over the course of the test, this creates a 'wave' of resource allocation and deallocation, which can help reveal elusive issues.

b. Stress/Break Test
This test uncovers your system's breaking point, which is useful for understanding the system's behaviour as it approaches failure. Such a test can be highly beneficial for technical stakeholders, ensuring that alarms and monitoring systems function as intended during critical situations. Start with a combined test, then once all users have been active for 30 minutes, incrementally add 50% more users. Repeat this process until the system fails under load. This strategy helps identify the maximum load the system can handle before experiencing significant performance degradation or outright failure.

As the business changes your system behaviour will also change. It is important to review on a 3 month basis any shift in the types or volume of transactions against your test model.

Conclusion

In closing, it's vital to understand that performance testing is not just a one-time activity. It is a continuous process, part of the life cycle of system development and maintenance, integral to the successful operation and improvement of your technical infrastructure. With the strategies and steps discussed in this article, you should be well-equipped to turn the daunting task of performance testing into a manageable, even streamlined, process.

Through our exploration of the purpose of performance testing, defining clear goals, aligning test environments, building production load models, aligning with SLAs, and finally developing, debugging, and combining test scripts, we've set up a comprehensive framework. This approach addresses the common pain points faced in this field, ensures the reliability of results through repetition, and aligns the test outcomes closely with real-world production scenarios.

Furthermore, the additional strategies, like the 72-hour Soak Test and Stress/Break Test, offer significant insights into your system's long-term stability and stress limits, proving invaluable in planning and resource allocation. Regular review of the test model helps keep our testing relevant and aligned with evolving business needs.

Remember, the ultimate aim of performance testing is not just about finding and fixing issues but about proactively improving system performance, efficiency, and robustness. This approach not only helps identify and resolve current performance concerns but also establishes a reliable system for ongoing performance monitoring and future problem anticipation.

In conclusion, this approach removes some of the complexities of performance testing. By fostering a culture of continuous improvement and in-depth understanding of system behavior, we can realize a more resilient, efficient, and sustainable system that can adeptly respond to current demands and future challenges.

Rethinking Project Management Tools: Our Switch from Trello to MS Planner

gallau — Sun, 04 Jun 2023 01:40:55 +0000

Several years ago, I worked at a company where we relied heavily on Trello for team management. At the time, there was a significant lack of organization in the tool's usage, with team members accessing it via personal user accounts. As the usage of the tool expanded with the addition of new projects, we did see a beneficial increase in effective communication. However, this expansion also brought to light several significant risks.

The first alarm bell rang when an owner of a critical board was on the verge of leaving the organization. This situation made us realize that several contractors, some of whom were no longer engaged with us, still had access. In some cases, these contractors were operating their own boards. We also discovered that some bug tickets contained data that should not have been stored outside the organization's secured systems. Moreover, personal accounts did not adhere to our organization's password management standards. This lack of compliance posed a risk of potential breaches, especially if these passwords were reused on other platforms that could be compromised.

Given these risks, we devised a solution with the following steps:

We identified all active boards and determined their specific purposes.
We chose Microsoft Planner as our new project management tool, given its similar functionality to Trello and its integration with our Teams application.
We informed the person responsible for Teams within the organization about our plans, highlighting the potential risk reassessment and the need for additional training and support.
We designed a decommissioning plan to archive data and reach out to current and past owners to delete their content.
We exported the boards to JSON files for future reference, handing them over to the MS Teams application owner for safekeeping.
We selected a migration tool, in this case, apps4.pro, to facilitate the transition from Trello to MS Planner.
Upon completion of the migration, we communicated the outcome to all stakeholders.
We updated our onboarding materials on the company wiki, noting the switch to MS Planner from Trello and summarizing the reasons behind the change.

Following our successful migration to MS Planner, we noticed substantial improvements. It was simpler to integrate with our internal applications, leading to a noticeable boost in productivity. The increased tool usage resulted in reduced cost per user, enhancing overall efficiency.

While Trello is undoubtedly a valuable tool, it requires careful management to prevent inadvertent exposure of the organization to potential risks. If organizational policies are implemented in the tool's usage, it can be better aligned with the company's risk appetite. With careful planning and a strategic approach, we were able to navigate these challenges and ensure a smooth transition to a more secure, productive project management system.

How to Automated a Business Process Using RPA

gallau — Wed, 31 May 2023 07:21:49 +0000

Recently, I was asked to assist one of the back-office administration teams struggling with time-consuming, repetitive tasks. These tasks, while essential, were prone to human error and involved data entry, data validation, and system integration activities, among others.

In assessing these processes, I recommended implementing Robotic Process Automation (RPA)—often referred to as RPC—as a proof of concept. RPC is a technology that emulates user behavior through scripts, allowing interaction with desktop applications or websites. The primary advantage is its ability to free up personnel for higher-value tasks, while the computer efficiently completes essential tasks.

To help our back-office administration team overcome their challenge, we opted for Microsoft's Power Automate Desktop (PAD). PAD is an end-to-end automation solution designed for a wide range of tasks.

If you're using Windows 10, PAD can be downloaded for free from the Microsoft store. However, be aware that your corporate policies might prevent you from installing it.

As a veteran in automating testing scripts, I adopted a similar process here. Here are the steps I followed:

Understand Your Objective and Calculate the ROI: Before embarking on any automation journey, it's crucial to identify why you're doing it and what you hope to gain. Are you trying to free up staff time? Improve accuracy? Increase speed? Having a clear objective will guide your process and help you make key decisions along the way. Next, assess the potential Return on Investment (ROI). Consider the amount of time and resources that will be spent on automating the process versus the time and cost savings the automation will provide once implemented. For example, if it's going to take two months to automate a process that is used only twice a year, the ROI may not justify the effort and expense. However, if a task is performed daily and requires several hours of manual work, the ROI from automation could be significant. Remember, automation is not always about direct financial returns—it can also provide value in the form of improved accuracy, consistency, and employee satisfaction.
Document the Process: Thorough documentation is a prerequisite for successful automation. The first step involves detailing every aspect of the process you plan to automate, ensuring nothing is left out. Capture all interactions, decision points, and exceptions. This documentation should serve as the blueprint for your automation design.

Recording a video of the process being executed manually can be an excellent tool for this. A video can provide more context than a textual description, illustrating nuances like where and how long the system pauses, how users navigate menus, and the precise sequence of inputs. This visual aid can be particularly helpful when you're dealing with complex systems or processes.

Once you've documented the process thoroughly, review it with the end users and stakeholders involved in the process. They can provide critical insights and validate the documented steps, ensuring that you've captured all the details accurately. This step is crucial because even minor oversights can result in automation failures or inefficiencies.

Lastly, remember that this document should be dynamic—update it as you make changes to the automation or as the underlying manual process evolves. This way, your documentation will always be in sync with the actual automated process.
Account for One-Time Pins and Two-Factor Authentication: These security measures can add a layer of complexity to automation. However, automation accounts can often be configured to bypass two-factor authentication or set to use a constant one-time pin (OTP). This can simplify the process significantly. However, it's essential to balance convenience with security, so implementing alternative security measures might be necessary to mitigate potential risks. For instance, this could include IP whitelisting, secure network access controls, or additional encryption for sensitive data. Always consult with your IT security team to ensure any changes adhere to your organization's security policies and standards.
Engage with the End User: Communication is crucial during the automation process. Regular catch-ups with the user who will be using the script can lead to beneficial suggestions and innovative solutions. In our case, these daily discussions paid off significantly. The business user suggested generating built-in reports to identify the records that needed processing. This insight allowed us to circumvent the complexities of Optical Character Recognition (OCR) on screen objects, simplifying our automation workflow. This real-world example underscores the value of maintaining open lines of communication with your end users—they often have intimate knowledge of the system and can provide valuable insights to improve the automation process. So, don't forget to include them in your automation journey.
Identify Repeatable Steps: As you're documenting the process and planning your automation, pay special attention to any steps that are repeated frequently. These could be as simple as logging into a system, navigating menus, or filling out a form. In Power Automate Desktop, you can create subflows for these repetitive tasks.

A subflow is like a function or subroutine in programming: it's a series of steps that you can define once and then call multiple times from different parts of your main flow. Not only does this save you the time of having to recreate these steps every time they're needed, but it also makes your main flow cleaner and easier to understand. Plus, if you ever need to change the way one of these repeated tasks is performed, you can just update the subflow, and the changes will be applied everywhere it's used. In our project, one such subflow was created to terminate the application with a DOS command line. This was useful in maintaining state without interacting with the UI. This approach reduces dependence on UI, which may be more prone to changes and inconsistencies, thus making your automation more reliable and easier to manage.

Remember, creating reusable components in your automation workflows, like subflows, is a best practice and significantly contributes to the efficiency and maintainability of your automation projects.
Leverage DOS and AI: When automating tasks, it's often more efficient and reliable to work with underlying data and systems directly, rather than through their user interfaces. For instance, rather than automating mouse clicks and keystrokes to copy data from one application and paste it into another, you could use DOS commands to extract the data from its source, transform it as needed, and load it into the target system. This approach, known as Extract, Transform, Load (ETL), is less prone to error and more resistant to changes in the user interface. It's important to note, though, that using DOS commands effectively requires a good understanding of command-line tools and scripting.
Artificial Intelligence (AI) can also be a significant ally in your automation efforts. Various AI models and tools can assist with different aspects of the automation process. For instance, AI can be used for writing regex expressions, identifying useful functions within the tool, and even troubleshooting issues you may encounter. Some AI models, like ChatGPT, Bing, Bard, or POE, are adept at text analysis and generation, while others can handle tasks like image recognition or prediction. By integrating AI into your automation workflows, you can solve more complex problems and create more intelligent, adaptable automation.
Break Down Your Process into Separate Flows: As you move forward with your automation project, consider breaking down your overall process into separate flows, especially if the process is complex or lengthy. Each flow should accomplish a distinct task or stage of the process. This approach can help manage complexity, enhance readability, and make troubleshooting easier.
For example, in our project, we divided the process into three distinct flows: extraction, transformation, and processing. The first flow handled the extraction of data, the second one transformed that data as necessary, and the third one processed the transformed data.

To share data between these separate flows, we utilized write-to-file functions. Data was written into files by one flow and then read from those files by the next. This way, even if one flow failed or needed to be rerun, it didn't necessarily impact the others. Each flow could be tested and debugged independently, which made the development process more manageable.

This modular approach to automation is an excellent practice. It promotes reusability, where the same flow could be used in different processes, and maintainability, as it's easier to update or fix a specific flow rather than a large, monolithic one. Keep in mind that organizing your automation into separate flows may require more initial planning and setup, but the payoffs in terms of scalability and manageability are well worth it.

Implementing this methodology, I created a series of three distinct flows, each tailored to a specific phase of the process: data extraction, data transformation, and data processing.

The first flow, data extraction, was designed to pull all the necessary data from the original sources. It was configured to access different databases or files, extract the required information, and store it for the next stage. This flow eliminated the manual need to gather data from various sources, thus minimizing the risk of human error and boosting efficiency.

The second flow, data transformation, took over where the first left off. It was responsible for manipulating the extracted data to meet the needs of the final processing stage. Depending on the task at hand, the transformation could involve operations such as data cleansing, format standardization, and the calculation of new data fields. Automating this step ensured that the data was consistently prepared and formatted correctly, saving time and reducing the likelihood of mistakes.

The third and final flow was tasked with data processing. It used the cleaned and transformed data to perform the necessary operations, such as updating records, generating reports, or performing complex computations. This final step replaced the time-consuming manual effort with a swift, accurate automated process.

The outcome of this methodical approach was significant. We successfully automated tasks equivalent to two man-days per month, freeing up valuable time for the team to focus on more strategic, value-adding activities.

Moreover, this project had an unexpected, but very welcome, ripple effect. The robust and flexible framework we developed served as a catalyst for further automation within the admin team. Seeing the efficiency and accuracy gains from the initial project, they started utilizing the framework to automate other manual processes in their workflow. This additional benefit underscored the transformative power of well-planned and executed automation, boosting productivity and reducing error rates beyond the initial scope of our project.

In conclusion, Robotic Process Automation using tools like Power Automate Desktop can significantly streamline business operations and drive efficiency. Through the implementation of this technology, we managed to automate tasks equivalent to two man-days per month of manual processing.

The exciting part about this journey is that once the initial framework was developed, the admin team began using it to automate other manual processes. This cascading effect of automation empowers teams to create a more efficient work environment.

However, it's important to remember that automation is not a one-size-fits-all solution. The process of implementing automation requires a clear understanding of the tasks at hand, a careful analysis of the return on investment, and a good degree of collaboration with the end-users. Also, consider the complexities that may arise, such as dealing with security measures like two-factor authentication, and the need for alternate methods to access and process data.

Breaking down the process into manageable pieces and documenting each step thoroughly can go a long way in ensuring the success of your automation project. Remember to leverage the power of technologies such as DOS and AI to handle complex tasks and make your automation more robust and intelligent.

Automation is an ongoing journey. The initial setup can take some time, but once the foundation is laid, the potential for scale and efficiency is remarkable. It opens the door to automating more complex and higher-value tasks, constantly raising the bar for what can be achieved.

Embrace the journey and remember that every challenge is an opportunity for growth and learning. With each hurdle, you gain knowledge and expertise that you can apply to future automation projects. Happy automating!

Mermaid: A Better Way to Version Diagrams

gallau — Sat, 13 May 2023 03:42:45 +0000

I recently joined a software team that used draw.io to create diagrams. The team was having trouble versioning their diagrams in git because draw.io diagrams are not text-based and cannot be easily committed to a repository. I suggested Mermaid as a better alternative to draw.io.

Mermaid is a tool that allows you to create diagrams and visualizations of your code. It is a popular tool among software developers because it can be used to represent a wide variety of code structures, including:

Class diagrams
Sequence diagrams
State diagrams
Flowcharts
Mind maps

Mermaid diagrams are easy to create and can be embedded in documentation, README files, and other text-based documents. They can also be exported to a variety of formats, including PNG, SVG, and PDF.

One of the benefits of using Mermaid is that it is text-based. This makes it easy to version Mermaid diagrams in git. Mermaid diagrams can be committed to a repository just like any other text file. This makes it easy to track changes to diagrams and to collaborate with others on diagramming projects.

I presented the team with the following benefits of using Mermaid:

Increased readability: Mermaid diagrams make it easier to understand complex code structures.
Improved communication: Mermaid diagrams can be used to communicate code to non-technical stakeholders.
Reduced errors: Mermaid diagrams can help you to identify and fix errors in your code.
Increased productivity: Mermaid diagrams can help you to save time by automating the process of creating documentation.
The team was convinced of the benefits of using Mermaid and decided to switch from draw.io to Mermaid. The team has been using Mermaid for several months now and has been very happy with the results. Mermaid has made it easy for the team to version their diagrams, collaborate on diagramming projects, and communicate code to non-technical stakeholders.

If you are looking for a better way to version diagrams, I recommend Mermaid. Mermaid is a powerful tool that can help you to improve the readability, communication, and productivity of your diagrams.

Here are some additional tips for beginners who are new to Mermaid:

The Mermaid syntax can be a bit daunting at first, but there are many resources available to help you learn it.
Once you have learned the basics, you can start creating your own Mermaid diagrams.
There are many examples of Mermaid diagrams available online, which can help you get started.
Mermaid is a powerful tool, but it is important to use it wisely. Make sure that your diagrams are clear and easy to understand.

I hope this helps!

Performance Testing Concepts & Terms

gallau — Sun, 23 Apr 2023 03:46:42 +0000

This article describes key concepts in building an approach to Performance Testing. Performance testing involves evaluating how a system performs under a specific workload. This can help determine if the system is able to handle the expected number of transactions and if its configuration is suitable.

Key Concepts

Load testing involves putting a system or application through a defined transaction model, either through scripts or by simulating a large number of users. The model typically consists of various functions, such as common transactions or transactions that require a high amount of system resources, such as searches and report retrieval.

Stress testing involves putting a system under abnormal or extreme workload in order to observe its behaviour. This can help identify if the system will crash or experience timeouts when subjected to high levels of transactions. For example, stress testing might involve seeing if a database connection times out before the application crashes when subjected to a high volume of transactions.

Response Times

These are the duration it takes to complete a transaction. There are different levels of transactions For user-facing transactions

<pre class="mermaid">
graph LR
    A[Start] --> B[Step 1]
    B --> C[Step 2]
    C --> D[Step 3]
    D --> E[End]
</pre>

<pre class="mermaid"> graph LR A[Start] --> B[Step 1] B --> C[Step 2] C --> D[Step 3] D --> E[End] </pre>

Maximizing Your Job Search Resources: Tips and Supports

gallau — Tue, 07 Feb 2023 10:23:00 +0000

Recently, I have noticed numerous LinkedIn posts regarding job redundancies. As someone who has experienced this situation a few times in my career, I would like to share some tips to help you in your search for a new role. Please keep in mind that these suggestions are specifically geared towards the Australian job market.

Why did you leave your previous role?

It's important to understand that the redundancy affected your role. The company's decision to discontinue the role was due to circumstances beyond your control and does not reflect on your performance in that role. Simply put, the role itself is no longer needed.

A common response to the question "What happened to your previous role at [company name]?" could be, "The role I held at [company name] was made redundant due to changes in the business environment. My current focus is to secure my next opportunity and continue my career growth." This concisely explains the situation and emphasizes your forward-looking perspective.

Preparing Your Tools and Supports

In your job search, you have access to various resources.

Employee Assistance Program (EAP): Some companies offer confidential counselling services through an EAP program. You can check with your HR department to see if your company subscribes to one of these programs. You are going through a big change to your life

LinkedIn: Reach out to former coworkers and business partners. They have their own professional networks and may know of job openings that match your skills and experience. Make sure to connect with them on LinkedIn, even if you did not previously do so. Ensure that your LinkedIn profile reflects the key points in your resume. You may have several resumes tailored to various positions but your LinkedIn profile should also echo these points.

Placement Services: Depending on your employer, you may be eligible for placement services that can help you update your resume, sharpen your interview skills, and negotiate salaries. This can be especially useful if you have been with the same company for a long time, as the recruitment process can change over time.

Professional Headshot: In certain industries, a professional headshot serves as a visual representation of your personal brand. With LinkedIn being a primary platform for job search and self-promotion, a professional headshot can help you stand out from other candidates and show that you take your work seriously. According to LinkedIn's own research, profiles with a professional headshot receive 21 times more views compared to those without.

A 30-second "elevator pitch": is a concise summary of your experience and skills. It gives recruiters a quick overview of who you are. Crafting a well-prepared pitch in advance can help you present yourself in a professional and succinct manner, effectively communicating the information you want to share.

A 60-second "elevator pitch": can be a crucial aspect of a job interview. The question "So, tell me about yourself" is frequently asked by 80% of interviewers, making it an excellent opportunity to highlight your most valuable skills and accomplishments. To make a strong impression, it's crucial to tailor your response to the target audience, showcasing information that aligns with their interests and the needs of the role. Your pitch should be concise and attention-grabbing, with a duration of between 30-60 seconds. Speak with confidence and allow your pitch to naturally flow before bringing your answer to a close.

Resume: Your resume may not accurately reflect your current skills and career goals. It's essential to update it to reflect the person you are today and the type of roles you are interested in pursuing. Rewriting your resume is the best way to achieve this. You can choose to hire a professional resume writer or use one of the many available online services like TopResume, Novoresume, or Zety to create a new resume. Make sure to customize your resume for the target market to increase its impact. A well-written resume should be brief and focused (3 pages or less), highlighting the key skills relevant to your desired role. An easy-to-read resume that directly addresses the selection criteria of the role can make the hiring decision easier and increase your chances of advancing to the next stage. You should have a core resume and then use it as a foundation to tailor your resume for other roles.

How to intergrate Chatgpt into VSCode IDE

gallau — Sun, 01 Jan 2023 05:27:56 +0000

I've been exploring what ChatGPT can do, and I'm currently learning the Go programming language. I thought it might be helpful to use ChatGPT as a resource while I'm learning Go. ChatGPT is a large language model trained by OpenAI and has a lot of knowledge about various topics, including Go programming. If you have any questions about Go or need help with a specific aspect of the language, ChatGPT can provide explanations and guidance. Please note that ChatGPT is not able to execute code or directly interact with your code editor, but it can offer helpful insights as you work on your Go projects.

I initially used the ChatGPT UI to interact with the model, but I found it inconvenient to copy and paste text between applications. That's when I discovered the ChatGPT Helper VSCode extension. It gives me the ability to:

Ask why my code isn't working
Request explanations for my code
Pose a free-form question

This extension has made it easier for me to use ChatGPT as I work on my Go projects.

Setup

Setting everything up is easy. Here's what you'll need:

A ChatGPT account
VSCode installed on your machine
ChatGPT Helper
Once you have all these items, you'll need to create an OpenAPI key. To do this, go to https://beta.openai.com/account/api-keys and click on "Create new secret key."

Copy the key and open VSCode. On a Windows machine, you can open the command palette by pressing the Ctrl + Shift + P keys. On a Mac, use the Command + Shift + P keys. This will bring up a prompt where you can paste the key.

To use ChatGPT, you'll need to enter a text, like "hello world." You'll then be asked for the API Secret key that you created earlier. Paste the key into the prompt and press Enter.

Next, you can use the command palette to ask a question. On a Windows machine, you can open the command palette by pressing the Ctrl + Shift + P keys. On a Mac, use the Command + Shift + P keys. This will bring up a prompt where you can enter your question.

Examples

Below are a few examples of ChatGPT's Helper capabilities in Go.

Ask why your code's not working?

package main


func main() {
    fmt.Println("hello world")
}

Below is the output

Ask to explain your code

package main

import "fmt"

func main() {
    fmt.Println("hello world")
}

Below is the output

Ask a free-form question

And below was the output

<code>package main

import (
    "fmt"
)

func main() {

    fmt.Println("Hello, playground")

    for i := 0; i &lt; 100; i++ {

        fmt.Println("Hello, playground")

    }
}
</code>


A:

You can use <code>strings.Repeat</code>: https://play.golang.org/p/RQx6z0J8-q3.  It's not the most efficient way to do it, but it's a simple way to do it.  If you want to be more efficient, you can use <code>bytes.Buffer</code> and <code>io.CopyN</code>.  See https://play.golang.org/p/hZj_5YtX-q3 for an example of that approach.  The second approach is more efficient because it doesn't have to allocate a new string each time through the loop, but the first approach is simpler and easier to understand if you're just getting started with Go.  If you're going to be doing this a lot, then the second approach is probably better, but if you're just doing it once or twice in your program then the first approach is probably better because it's simpler and easier to understand and maintain.  The second approach is also more complicated because it uses <code>io</code>, which is a package that deals with input and output (I/O), which can be confusing for beginners because I/O can be complicated and confusing in general (and especially in Go).  The first approach doesn't use I/O at all, so it's simpler in that regard as well as being simpler overall (in my opinion).  But again, if you're going to be doing this a lot then the second approach might be better because of its efficiency (but only if efficiency matters).  If efficiency doesn't matter then the first approach is probably better because of its simplicity (again, in my opinion).

Conclusion

In conclusion, the above post provides helpful information for getting started with ChatGPT. It's clear that ChatGPT can be a valuable resource for learning Go programming and improving your understanding of various concepts. It would be interesting to hear about other ways that people are using ChatGPT to enhance their learning experiences. Overall, ChatGPT seems like a useful tool for anyone looking to improve their skills and knowledge in the field of programming.

How to integrate OpenAPI Codex into VSCode IDE

gallau — Tue, 27 Dec 2022 04:03:04 +0000

Thank you to @tyler36 who pointed out in the repo for Chatgpt helper
it states "This is an extension to quickly query OpenAI's Codex, an AI code assistant similar to ChatGPT, from VS Code.

Apologies for the mistake

Note:
Temporarily, the extension is using Codex, a similar model by OpenAI, instead of ChatGPT, since ChatGPT is near impossible to automate at this time."

Please subsite OpenAPI Codex for ChatGPT.

Ask why my code isn't working
Request explanations for my code
Pose a free-form question

This extension has made it easier for me to use ChatGPT as I work on my Go projects.

Setup

Setting everything up is easy. Here's what you'll need:

To use ChatGPT, you'll need to enter a text, like "hello world." You'll then be asked for the API Secret key that you created earlier. Paste the key into the prompt and press Enter.

Examples

Below are a few examples of ChatGPT's Helper capabilities in Go.

Ask why your code's not working?

package main


func main() {
    fmt.Println("hello world")
}

Below is the output

Ask to explain your code

package main

import "fmt"

func main() {
    fmt.Println("hello world")
}

Below is the output

Ask a free-form question

And below was the output

<code>package main

import (
    "fmt"
)

func main() {

    fmt.Println("Hello, playground")

    for i := 0; i &lt; 100; i++ {

        fmt.Println("Hello, playground")

    }
}
</code>


A:

You can use <code>strings.Repeat</code>: https://play.golang.org/p/RQx6z0J8-q3.  It's not the most efficient way to do it, but it's a simple way to do it.  If you want to be more efficient, you can use <code>bytes.Buffer</code> and <code>io.CopyN</code>.  See https://play.golang.org/p/hZj_5YtX-q3 for an example of that approach.  The second approach is more efficient because it doesn't have to allocate a new string each time through the loop, but the first approach is simpler and easier to understand if you're just getting started with Go.  If you're going to be doing this a lot, then the second approach is probably better, but if you're just doing it once or twice in your program then the first approach is probably better because it's simpler and easier to understand and maintain.  The second approach is also more complicated because it uses <code>io</code>, which is a package that deals with input and output (I/O), which can be confusing for beginners because I/O can be complicated and confusing in general (and especially in Go).  The first approach doesn't use I/O at all, so it's simpler in that regard as well as being simpler overall (in my opinion).  But again, if you're going to be doing this a lot then the second approach might be better because of its efficiency (but only if efficiency matters).  If efficiency doesn't matter then the first approach is probably better because of its simplicity (again, in my opinion).

Conclusion

PS ChatGPT may of helped with the writing

Learning Go

gallau — Sun, 11 Dec 2022 01:33:27 +0000

Go-lang was created at Google in 2007, at the time of writing this post Go is at version 1.19. I have been learning to program with Go to create AWS Lambdas to offload some admin tasks.

As I am from a Performance Testing background and have used Loadrunner which uses a C variant compiler I am familiar with some programming practices. However, Go was a whole new world for me.

The following resources I am currently using to aid my learning. This article is aimed at those looking to get into Go programming.

Podcast: Go Time

"Your source for diverse discussions from around the Go community. This show records LIVE every Tuesday at 3 pm US Eastern. Join the Golang community and chat with us during the show on the #gotimefm channel of Gophers slack. Panellists include Mat Ryer, Jon Calhoun, Natalie Pistunovich, Johnny Boursiquot, Angelica Hill, Kris Brandow, and Ian Lopshire. We discuss cloud infrastructure, distributed systems, microservices, Kubernetes, Docker… oh and also Go! Some people search for GoTime or GoTimeFM and can’t find the show, so now the strings GoTime and GoTimeFM are in our description too."

I googled learning go and the "Go Times" Go for Beginners episode appeared on the list. The guests on the show highlighted several resources across the web and provided general advice.

Linkedin Learning

A friend advised me earlier this year that you can access Linked Learning for free at the local library. So if your library offers you access to LinkedIn Learning I would recommend Go Essential Training. This is aimed at a beginner level and will teach you how to set up your development environment. Thank you Claire for the tip.

Exercism

I wanted to use this site as it was recommended by the Go Time podcast. It was above my level at the time. I found the interface confusing but after the first couple of challenges, I started to get into a rhythm. I found it easier to complete the challenges when I pulled the examples of git into my local VScode IDE.

Learn Go with Tests

As I have a background in testing I found this course to be the most straightforward. It walks you through the TDD process. This involves writing your tests first and then your code. As I had completed the previous courses this course reinforced my learning.

Learn X in Y minutes

If you have completed the above courses or you have a knowledge of programming this class is very beneficial for revision. It is nicely laid out on a single page and uses comments to guide you through the various go practices.

So hopefully the above suggestions will assist you on your journey. If you are brand new to coding I would recommend looking at a beginner course on git. This will make it easier to manage all the files that you create.

If you know of other materials that could be helpful can you please leave a comment with a link?

Thank you.

Best of luck

Is mise

Gallau

YOW! Conference Melbourne 2022 what I learned

gallau — Sat, 10 Dec 2022 07:48:26 +0000

I attended YOW! Melbourne 2022 conference this week. I was looking forward to learning about other perspectives on their experience of developing and supporting systems, and the conference delivered.

YOW! Describe themselves as:
"YOW! Conferences bring together internationally recognized speakers and developers for an in‑depth look at emerging technologies and best practices in the software industry – regardless of technological platform or language — without commercial hype."

My organisation was one of the sponsors and I had the opportunity to staff our booth. This had the added benefit of meeting more people and learning more about other tech organisations than if I was just attending the conference. In the future, if your company is one of the sponsors I would recommend staffing their booth. This will help refine your public speaking skills and increase the number of industry connections.

These were some of the sessions I attended:

A Kafkaesque Series of Events by Lily Mara
Managing your SLO in the face of Chaos by Liz Fong-Jon
Sensible defaults for tech management by Michelle Gleeson

One of the themes I noticed in a lot of the talks was the theme of test and learn. I enjoyed listening to WAR Stories and Lily Mara's "A Kafkaesque series of events"; Liz Fong-Jones's "Managing to your SLO in the face of Chaos" did not disappoint. My takeaways from both of their presentations were to be aware of the context of the problem and to evolve your ways of working as the system grows. Given the massive volumes of traffic, both OneSignal and Honeycomb manage daily. It was an interesting insight into the importance of observability and clear communication with all your stakeholders. This is the key to growing your business.

The most significant item I learned from the conference was the number of Female Senior Engineers who leave our industry after 10 years due to a lack of promotion and investment. Michelle Gleeson outlined her lived experience and shared some horrific workplace experiences and also some ways to combat the issue.

Some of my company's fellow Engineering Managers also attended this talk and it sparked some discussion on how we can start to develop and change practices to ensure we do not lose talented engineers to the same issue and how we can make this a key strength for our company in a tight hiring market.

Michelle is one of the co-founders of Tech Leading Ladies a meetup and coaching group, this community is designed to break down the barriers to promotion for women in Tech.

Speaking to other attendees at the booth or on the conference floor there was a theme of all companies attempting to work in a hybrid world. Most progressive companies hold an office day for the team to meet face-to-face. Meetings focused on planning and problem solving, with the remainder of the sprint focused on delivering and staff deciding where they needed to work.

Another theme was what was coming in 2024. The tech hiring freezes and redundancies reported overseas had put some Australian companies in a more cautious frame of mind. Others spoke about aggressive hiring targets for 2024.

If I see any links to the above talks I will post them later.

PS A tip, the mech moves fast, get in early to the booths