It's the end of Sprint 4. I'm happy to say that I finally find my groove working on Hurl. And as usual, I also did some chores on the Starchart project.
🦀🌐 Hurl
This week I've been working towards a feature request to enable queries on HTTP redirects. Same as before, every time I worked on something new, I was faced with all sorts of confusions. But this time I decided to do more code reading and try to understand it even more.
Code Reading
I started with a similar PR, hoping to find some inspiration. Then I quickly realized this PR only focuses on evaluating the queries, and this feature goes beyond that.
Instead, I followed my intuition to a file I already worked on - the HTTP client file hurl::http::client. I wasn't sure if it's related, but I knew that it was involved in this request and response process. Using the IDE, I followed the trail of where each function was used and eventually constructed a few diagrams to help me understand things:
The diagram above illustrates the basic process of everything involved from the moment when the Hurl file is parsed, to when HTTP requests are made. The green blocks represent function calls, and the red ones are for return values.
Hurl files are first parsed into entries. Each entry contains one call if there's no redirections, or multiple calls if there is. And each call contains a pair of HTTP request and response.
Hurl uses the final call in Vec<Call> to further evaluate information for captures and asserts. The diagram above only shows the capture evaluation process, but assert is similar.
This is a (partial) diagram of the data structures involved in the previous process.
Working on the Feature (Link to PR)
In order to evaluate redirections, I need the knowledge of all the HTTP requests made in the whole request chain. Looking at the diagrams, the last place with the said knowledge is entry::run - since Vec<Call> is returned to this function, and each Call only contains one request. In other words, entry::run is probably where I should work on.
Looking at the code, the actual queries are not evaluated until query::eval_query, but the entire chain of Vec<Call>.last() -> http::Response -> response::eval_captures -> capture::eval_capture -> query::eval_query only knows the final HTTP request. I had to figured out a way to pass the knowledge of others down the chain. To achieve this, I thought of 2 solutions:
Comment for
#922
@jcamiel Hello! I dug into the code quite a bit. Right now, because our response::eval_captures and response::eval_asserts methods accept one HTTP request, we only use the response from the last call:
https://github.com/Orange-OpenSource/hurl/blob/95a2cc55dd318a0a24ac0f3c88245883878853a9/packages/hurl/src/runner/entry.rs#L130-L132
Since we need all the HTTP requests in order to retrieve the redirects, I can think of two solutions to this. Let me know which one you prefer (or something else):
- Make a helper function to extract all the HTTP responses from a list of calls, and modify
response::eval_capturesandresponse::eval_assertsto accept&[&http::Response]instead. - Modify
http::Responseto include a list of redirections (or a list of references to the previous responses?)
Really appreciate your input 🙏
The maintainer picked the first one and gave me some extra help. And that was all I needed to work on my first PR!
Clippy Corner
I'm officially making this a segment because every time I work on Rust, clippy always finds a way to improve my code.
This time, I got a warning of #useless_vec due to the usage of &vec![...]
This is useless because the reference operator & implies that I don't need ownership of this vector. And If I don't need ownership, there's no need to use a vector at all! It's a waste of resources. Instead, I should use a slice: &[...]
⭐🗺️ Starchart
This week's work on Starchart is a mix of housekeeping and research.
Update Dependencies (Link)
It's only been two weeks since my last "update dependencies" PR, but we were swarmed by another wave. This time, I upgraded 18 packages without any breakages.
I was stumbled by a failing test cause by dependabot, but quickly figured out the problem: All remix packages need to be updated together, but dependabot tried to update just one.
Looking into MX Records
This one was tricky because the dev server is not deployed anywhere and it's hard to test public-facing services like DNS on a mock server.
With the help of AI (or rather, LLM 👀), I was recommended the aws client, and it worked!
Comment for
#852
@lilyhuang-github @uday-rana I was able to use the aws client to retrieve the DNS records from the motoserver/moto mock server:
Using this command:
aws route53 list-resource-record-sets --hosted-zone-id <zone-id> --endpoint-url=<url>
I got:
{
"Name": "mail.user1.starchart.com.",
"Type": "MX",
"TTL": 300,
"ResourceRecords": [
{
"Value": "10 mail.com"
}
]
}
Seems like the record is successfully created!
I did so by pulling up a shell inside the mock server's docker container and installing the aws client in there. Before you tell me this is super illegal, and I can just install the client on my native system, I did it this way because the client comes with a ton of dependencies and I don't really want to mess up my own system because of it. 😆
Investigating the samlify Breakage
This package has been broken since version 2.8.11 and we're on version 2.8.10 for now.
I decided upgrade this again and analyze the error message. Here's what I had on the server
Backtrace:
1 www/_include.php:45 (SimpleSAML_exception_handler)
0 [builtin] (N/A)
Caused by: Exception: Invalid value of boolean attribute 'AllowCreate': ''
Backtrace:
6 vendor/simplesamlphp/saml2/src/SAML2/Utils.php:282 (SAML2\Utils::parseBoolean)
5 vendor/simplesamlphp/saml2/src/SAML2/AuthnRequest.php:233 (SAML2\AuthnRequest::parseNameIdPolicy)
4 vendor/simplesamlphp/saml2/src/SAML2/AuthnRequest.php:168 (SAML2\AuthnRequest::__construct)
3 vendor/simplesamlphp/saml2/src/SAML2/Message.php:572 (SAML2\Message::fromXML)
2 vendor/simplesamlphp/saml2/src/SAML2/HTTPRedirect.php:125 (SAML2\HTTPRedirect::receive)
1 modules/saml/lib/IdP/SAML2.php:316 (sspmod_saml_IdP_SAML2::receiveAuthnRequest)
0 www/saml2/idp/SSOService.php:19 (N/A)
Although the error itself is not related to our project, the AllowCreate attribute caught my eye. I first searched for it within our code base. I found nothing there, so I decided to look upstream.
A quick search in samlify's repo brought me to this issue, and it seems like many people are experiencing this issue as well. Luckily, a fix is already on its way.
❤️🧠 A Quick Note on Stress
The past few weeks have been really difficult to navigate. Tasks from school are piling up, and the stress from job searching too. I was worried that I couldn't get much else done in the midst of all that. I guess this blog proved that wrong and I'm proud of it haha. Let's hope that I can keep this momentum forward!
Top comments (0)