Planet Scheme

Monday, November 22, 2021

Idiomdrottning

Email Netiquette

The most important rule for email is to live and let live.

Other people are gonna bork up the subject lines, CC you on list mail, be overly verbose or terse, top-post, bottom-post, interleave, fullquote, forget to quote, reply too quickly, too slowly, ask too many questions, too few questions, have annoying signatures etc etc etc. That’s fine.

Get and keep your own house in order with whatever filters and templates you need. Then don’t worry about it.

I love reading advice that’s about dealing with email, like

What I’m over, though, is advice that’s about how horrible everyone else is emailing.

We needed email but people got stressed out and they started flocking to these silo sites like Facebook and Twitter which have a more codified interaction pattern that enforces or rewards brevity, picture tagging, and event scheduling.

If we wanna get people back into email then we can’t be all shamey and gatekeepy about it.♥

by Idiomdrottning (sandra.snan@idiomdrottning.org) at Monday, November 22, 2021

Sunday, November 21, 2021

Idiomdrottning

Wrapping and raising

In the world of 3D-modeling, back in the day before sculpting and retopo were invented, there were two different styles of modeling. Point-by-point (where you create a point, join it up to your other points with a line, and join that line up with your other lines with a face) and box modeling, where you start with a box, and cut and push and add loops to it.

When I’m using paredit to work on Lisp code, I’m using it like a box modeler.

The core primitives for me are wrapping and raising.

I start by inserting an empty pair and as I write, I keep adding pairs in. In the following evolving example, the bar character | represents my cursor.

(|)

(let ((|)))

(let ((foo|)))

(let ((foo (|))))

(let ((foo (bar|))))

(let ((foo (bar (|)))))

(let ((foo (bar (+ 11 12|)))))

(let ((foo (bar (+ 11 12))))|)

(let ((foo (bar (+ 11 12))))
  (|))

(let ((foo (bar (+ 11 12))))
  (frobnicate foo foo|))

So far, so normal. Like any other paredit user. Now let’s say I later realize that I need one of those foos to be reversed.

Let me first show you what I would not do:

(let ((foo (bar (+ 11 12))))
  (frobnicate (|) foo foo))

(let ((foo (bar (+ 11 12))))
  (frobnicate (reverse|) foo foo))

(let ((foo (bar (+ 11 12))))
  (frobnicate (reverse foo|) foo))

Using a forward-slurp-sexp.

Instead, I’d start by wrapping the foo, and then adding the new operator.

(let ((foo (bar (+ 11 12))))
  (frobnicate (|foo) foo))

(let ((foo (bar (+ 11 12))))
  (frobnicate (reverse |foo) foo))

If I later decide I don’t want that foo reversed after all, I can use raise-sexp on it. Wrapping and raising are the core primitives of working with the syntax tree as a tree, always preserving the syntactic structure, as opposed to seeing it just as text that happens to have a matching paren that you want to push rightwards by slurping.

There’s also splicing and splice-killing that are useful.

by Idiomdrottning (sandra.snan@idiomdrottning.org) at Sunday, November 21, 2021

Saturday, November 20, 2021

Idiomdrottning

define vs let

Obviously let bindings are awesome because their scope is immediately clear as opposed to a define that’s gonna keep on cluttering the top level forever.

One neat thing about defines, though, is how they bind in two ways.

Obviously, you can think of the name itself as a binding, like how these two examples resemble each other (except for scope):

(let ((foo (+ 12 18)))
  ...)

(define foo (+ 12 18))
...

That’s basic. The next level is the function args are also implicitly bindings!

(define (bar foo)
  ...)

(bar (+ 12 18))

Now, if you already know Scheme, you’re gonna say “Duh, we know that”. But the neat part is applying this in the context of refactoring, inlining and extracting!

If you are an inliniac like me, you’re letting your program’s own evolution determine your abstraction levels, letting your own code show you which things need to be bound and which things don’t, instead of trying to decide that on your own.

Now, the ninja move is to let those bindings, along with any of your “zebra” bindings, live on the arg name level, like in the last example above.

Tired:

(let ((zebra (striped horse (donkey tail))))
  (when zebra (frobnicate leaves)))

Wired:

(when (striped horse (donkey tail))
  (frobnicate leaves))

Inspired:

(define (friendly zebra)
  (when zebra (frobnicate leaves)))

(friendly (striped horse (donkey tail)))

The awesome thing about this style is that this makes the binding itself the interface. This makes the code super flexible and reusable.♥

by Idiomdrottning (sandra.snan@idiomdrottning.org) at Saturday, November 20, 2021

Inliniac

I tend to love inlining and extracting. Whenever something is only used once, I wanna inline. When two things look the same (especially if they’re long), I wanna extract.

(let ((zebra (striped horse (donkey tail))))
  (foo
   (bar baz quux)
   zebra
   (bar baz quux)))

I would wanna turn into:

(let ((bbq (bar baz quux)))
  (foo
   bbq
   (striped horse (donkey tail))
   bbq))

Sometimes when I am collaborating with people on code, they want me to turn some of those striped horses into zebras, i.e. introduce some new semantic units for clarity and explicitness, and turn some of those repeated bbqs into bar baz quuz if the newly extracted semantic unit is overly abstract and indirect.

And that’s fine, it’s not like I don’t ever wanna compromise. I already have plenty of code in my preferred Idiomdrottning style, and I don’t see it as a huge loss of face to change some bindings around for my friend’s sake. I also sometimes for my own clarity’s sake do introduce some zebras (in the sense of strictly unnecessary semantic units), although I prefer to place them as defines, as their own functions, rather than as a let binding.

In other words, my style preference here isn’t an absolute.

Let’s get an obvious caveat out of the way. Yes, this style means that as the program evolves, I need to be willing to extract what I had previously inlined (if I need to use it elsewhere) or inline what I’ve previously extracted (if I end up only using it once). I always have the “inline”/“extract” perspective on.

That becomes especially necessary when you consider the following: We have a machine that can make popcorn and one for coffee. We see that some of the functions are similiar so we extract them to be the same function. If we later need up having to make the same change to both machines, we’re gonna be really happy that we extracted the part. We kept our code DRY and clean. However, if we end up having to make completely different changes one or both of the machines, we might need to start by inlining. If we always have “inline”/“extract” mindset on, we can spot those situations right away and deal with them. If we don’t, we are gonna suffer. It’s gonna be pretty rough trying to figure out how to add salt to your popcorn and not your coffee.

However, I want to explain a little bit where I’m coming from with this, what’s my motivation and what benefits I’ve found with my style.

This defers making choices about what abstractions to use. The program, as it evolves, creates those for us, just by us extracting what’s used twice or more, or inlining what’s only used once. Please don’t underestimate how awesome this is. It tends to create abstraction levels that are just right, just perfect. Not right away, but, as the program is being worked on and refactored and cared for in this way.

Instead of us trying to abstract a domain’s logic into code, this method just does that for us automatically. That is a wonder that never ceases to amaze me. Don’t worry about those extra bbqs. They tend to end up either sorting themselves out again, or, end up being exactly what the doctor ordered to make sense of the code. (Be careful when coming up with good names for them. As a personal practice, I always call them weird zorky names like “frobnicate” as a placeholder, some thing I know that I’ll replace. If they had been called SyncManagerHandler, that name might’ve stuck. Instead, I want a naming pass after I see what the function does.)

And then you can introduce your extra zebras to the extent that that helps clarity and matches domain expectations.

by Idiomdrottning (sandra.snan@idiomdrottning.org) at Saturday, November 20, 2021

Jérémy Korwin-Zmijowski

Guile Hacker Handbook - Strings

Guile Logo

A new chapter of the book has been published in the data types section: strings! The practical exercise is taken from programmingpraxis.com. The goal is to create a procedure that returns the longest duplication in a given string.

This is an opportunity to approach a technique called triangulation to generalize the behavior of a software.

I lead the readers on the first tests. Then I invite them to continue and share their work to receive feedback that would help them to improve!

Thank you very much for reading this article!

Don't hesitate to give me your opinion, suggest an idea for improvement, or ask a question! To do so : contact me.

Don't miss out on the next ones...

  1. articles via Mastodon @jeko@write.as and RSS
  2. screencasts via Peertube jeko@video.tedomum.net and RSS

And more importantly, share this blog and tell your friends it's the best blog in the history of Free Software! No kidding!

#handbook #guile #english

Saturday, November 20, 2021

Friday, November 19, 2021

Jérémy Korwin-Zmijowski

Guile Hacker Handbook - Chaînes de caractères

Guile Logo

Un nouveau chapitre du livre a été publié dans la section des types de données : les chaînes de caractères ! L'exercice pratique est tiré du site programmingpraxis.com. Le but est de développer une procédure qui renvoie la plus longue duplication dans une chaîne de caractères donnée.

C'est l'occasion d'aborder une technique appelée la triangulation pour généraliser le comportement d'un logiciel.

J'y accompagne le lecteur sur les premiers tests. Apres quoi je l'invite à poursuivre et partager son travail pour recevoir des retours qui l'aiderait à s'améliorer !

Merci beaucoup d'avoir lu cet article !

N'hésite pas à me contacter pour me donner ton avis, proposer une idée d'amélioration, ou poser une question !

Abonnes-toi pour ne pas manquer les prochains :

  1. articles via Mastodon @jeko@write.as et RSS
  2. screencasts via Peertube jeko@video.tedomum.net et RSS

Et encore plus important, partages ce blog et dis à tes amis que c'est le meilleur blog de l'histoire du logiciel libre ! Sans dec'

#handbook #guile #français

Friday, November 19, 2021

Tuesday, November 16, 2021

Scheme Requests for Implementation

SRFI 227: Optional Arguments

SRFI 227 is now in final status.

This SRFI specifies the opt-lambda syntax, which generalizes lambda. An opt-lambda expression evaluates to a procedure that takes a number of required and a number of optional (positional) arguments whose default values are determined by evaluating corresponding expressions when the procedure is called.

This SRFI also specifies a variation opt*-lambda, which is to opt-lambda as let* is to let and the related binding constructs let-optionals and let-optionals*.

Finally, for those who prefer less explicit procedure definitions, a sublibrary provides define-optionals and define-optionals*.

by Marc Nieper-Wißkirchen (spec and R6RS implementation) and Daphne Preston-Kendal (R7RS implementation) at Tuesday, November 16, 2021

Monday, November 15, 2021

Scheme Requests for Implementation

SRFI 230: Atomic Operations

SRFI 230 is now in final status.

This SRFI defines atomic operations for the Scheme programming language. An atomic operation is an operation that, even in the presence of multiple threads, is either executed completely or not at all. Atomic operations can be used to implement mutexes and other synchronization primitives, and they can be used to make concurrent algorithms lock-free. For this, this SRFI defines two data types, atomic flags and atomic (fixnum) boxes, whose contents can be queried and mutated atomically. Moreover, each atomic operation comes with a memory order that defines the level of synchronization with other threads.

by Marc Nieper-Wißkirchen at Monday, November 15, 2021

SRFI 229: Tagged Procedures

SRFI 229 is now in final status.

This SRFI defines tagged procedures, which are procedures that are tagged with a Scheme value when created through the syntax lambda/tag and case-lambda/tag. The value of the tag of a procedure can be retrieved with procedure-tag, and the predicate procedure/tag? discerns whether a procedure is tagged.

by Marc Nieper-Wißkirchen at Monday, November 15, 2021

Saturday, November 6, 2021

The Racket Blog

Racket v8.3

posted by John Clements

Racket version 8.3 is now available from https://download.racket-lang.org.

  • Racket removes syntax arming and disarming in favor of a simpler system of protected syntax operations, along with other updates to the syntax system.

  • DrRacket has improved support for custom #lang languages.

  • Typed Racket improves precision for type-checking of non-polymorphic structures, existential types, and certain binding forms.

  • Scribble HTML output gains a button to show / hide the table of contents on mobile platforms.

  • Redex’s stepper’s GUI shows IO-judgment form rule names.

  • Many bug fixes!

The following people contributed to this release:

Adam Zaiter, Alex Knauth, Alexis King, Ayman Osman, Ben Greenman, Bob Burger, Bogdan Popa, Brian Adkins, Cameron Moy, Carl Eastlund, Dan Holtby, Dominik Pantůček, Eli Barzilay, Ethan Leba, Fred Fu, Greg Hendershott, Gustavo Massaccesi, J. Ryan Stinnett, Jason Hemann, Jay McCarthy, Jesse Alama, Joel Dueck, John Clements, Jonathan Simpson, Kartik Sabharwal, Laurent Orseau, Lehua Ding, Maciej Barć, Marc Burns, Matthew Flatt, Matthias Felleisen, Michael Ballantyne, Mike Sperber, Noah Ma, Paulo Matos, Pavel Panchekha, Philip McGrath, Robby Findler, Ryan Culpepper, Ryan Sundberg, Sage Gerard, Sam Tobin-Hochstadt, Shu-Hung You, Sorawee Porncharoenwase, Stefan Schwarzer, Stephen De Gabrielle, Vincent St-Amour, William J. Bowman, minor-change, and yjqww6

Feedback Welcome

by The Unknown Author at Saturday, November 6, 2021

Tuesday, October 26, 2021

GNU Guix

From ‘guix environment’ to ‘guix shell’

There are times when what looked like the right design choice some years back comes out as an odd choice as time passes. The beloved guix environment tool is having that fate. Its command-line interface has become non-intuitive and annoying for the most common use cases. Since it could not be changed without breaking compatibility in fundamental ways, we devised a new command meant to progressively replace it; guix shell—that’s the name we unimaginatively ended up with—has just landed after a three-week review period, itself a followup to discussions and hesitations on the best course of action.

This post introduces guix shell, how it differs from guix environment, the choices we made, and why we hope you will like it.

The story of guix environment

The guix environment command started its life in 2014, when Guix was a two-year old baby and the whole community could fit in a small room. It had one purpose: “to assist hackers in creating reproducible development environments”. It was meant to be similar in spirit to VirtualEnv or Bundler, but universal—not limited to a single language. You would run:

guix environment inkscape

… and obtain an interactive shell with all the packages needed to hack on Inkscape; in that shell, the relevant environment variables—PATH, CPATH, PKG_CONFIG_PATH, and so on—would automatically point to a profile created on the fly and containing the compiler, libraries, and tools Inkscape depends on, but not Inkscape itself.

Only a year later did it become clear that there are cases where one would want to create an environment containing specific packages, rather than an environment containing the dependencies of packages. To address that, David Thompson proposed the --ad-hoc option:

guix environment --ad-hoc inkscape -- inkscape

… would create an environment containing only Inkscape, and would then launch the inkscape command in that environment. Many features were added over the years, such as the invaluable --container option, but these two modes, development and “ad hoc”, are the guts of it.

Fast forward six years: today, there’s consensus that the name --ad-hoc is confusing for newcomers and above all, that the “ad hoc” mode should be the default. This is the main problem that guix shell addresses.

Doing what you’d expect

Changing the default mode from “development environment” to “ad hoc” is technically easy, but how to do that without breaking compatibility is harder. This led to lengthy discussions, including proposals of mechanisms to choose between the new and old semantics.

In the end, keeping the guix environment name while allowing it to have different semantics was deemed dangerous. For one thing, there’s lots of material out there that demoes guix environment—blog posts, magazine articles, on-line courses—and it would have been impossible to determine whether they refer to the “new” or to the “old” semantics. We reached the conclusion that it would be easier to use a new command name and to eventually deprecate guix environment.

With guix shell, the default is to create an environment that contains the packages that appear on the command line; to launch Inkscape, run:

guix shell inkscape -- inkscape

The --ad-hoc option is gone! Likewise, to spawn an ephemeral development environment containing Python and a couple of libraries, run:

guix shell python python-numpy python-scipy -- python3

Now, if you want, say, the development environment of Inkscape, add the --development or -D option right before:

guix shell -D inkscape

You can add Git and GDB on top of it like so:

guix shell -D inkscape git gdb

(Note that -D only applies to the immediately following package, inkscape in this case.) It’s more concise and more natural than with guix environment. As can be seen in the manual, all the other options supported by guix environment remain available in guix shell.

Short-hands for development environments

A convention that’s become quite common is for developers to provide a guix.scm at the top of their project source tree, so that others can start a development environment right away:

guix environment -l guix.scm

The guix.scm file would contain a package definition for the project at hand, as in this example. This option is known as -f in guix shell, for consistency with other commands, and the equivalent command is:

guix shell -D -f guix.scm

Since all Guix commands accept a “manifest” with -m, another option is to provide a manifest.scm file and to run:

guix shell -m manifest.scm

“Wouldn’t it be nice if guix shell would automatically follow these conventions when not given any argument?”, some suggested. As in the case of Bundler, direnv, or typical build tools from Meson to Make, having a default file name can save typing and contribute to a good user experience for frequently-used commands. In this spirit, guix shell automatically loads guix.scm or manifest.scm, from the current directory or an ancestor thereof, such that entering a project to hack on it is as simple as:

cd ~/my/project/src
guix shell

Worry not: guix shell loads guix.scm or manifest.scm if and only if you have first added its directory to ~/.config/guix/shell-authorized-directories. Otherwise guix shell warns you and prints a hint that you can copy/paste if you want to authorize the directory.

Caching environments

With that in place, guix shell can pretty much fill the same role as direnv and similar tools, with one difference though: speed. When all the packages are already in store, guix shell can take one to a few seconds to run, depending on the package set, on whether you’re using a solid state device (SSD) or a “spinning” hard disk, and so on. It’s acceptable but prohibitively slow for direnv-like use cases.

To address that, guix shell maintains a profile cache for the -D -f guix.scm and -m manifest.scm cases. On a hot cache, it runs in 0.1 second. All it has to do is fork a shell with the right environment variable definitions; it does not talk to guix-daemon, and it does not even read guix.scm or manifest.scm (it’s possible to forcefully update the cache with --rebuild-cache).

That makes guix shell usable even for short-lived commands like make:

guix shell -- make

Hopefully it’ll change the way we use the tool!

The shell doctor

While revamping this command-line interface, the idea of a “shell doctor” came up. In interactive use, guix shell sets environment variables and spawns a shell, but it’s not uncommon for the shell to mess up with the whole environment. Why? Because, contrary to documented practice, it’s quite common for users to define or override environment variables in the startup files of non-login shells, ~/.bashrc for Bash, ~/.zshrc for Zsh. Instead, environment variable definitions should go to the startup file of login shells—~/.bash_profile, ~/.profile, or similar. But let’s face it: it’s a subtle distinction that few of us know or care about.

As a result, users of Guix, especially on distros other than Guix System, would often be disappointed when running guix environment --pure and yet find that PATH contains non-Guix entries, that there’s a bogus LD_LIBRARY_PATH definition, and whatnot. Now, they can call the doctor, so to speak, to obtain a diagnosis of the health of their shell by adding the --check flag:

guix shell --check python python-numpy

The command creates an environment containing Python and NumPy, spawns an interactive shell, checks the environment variables as seen by the shell, and prints a warning if PATH or PYTHONPATH in this case have been overridden. It does not tell users where the problem comes from—it cannot guess—but it tells them if something’s wrong, which is a first step.

Of course, the best way to sidestep these problems is to pass --container, which gives a fresh, isolated environment that does not contain those startup files. That’s not always an option though, for instance on systems lacking support for unprivileged user namespaces, so --check comes in handy there.

Try it!

Just run guix pull to get this shiny new guix shell thingie!

If you don’t feel ready yet, that’s OK: guix environment won’t disappear overnight. We have a written commitment to keep it around until May, 1st 2023. Though overall, we hope you’ll find the guix shell interface easier to use and compelling enough that you’ll be willing to switch overnight!

About GNU Guix

GNU Guix is a transactional package manager and an advanced distribution of the GNU system that respects user freedom. Guix can be used on top of any system running the Hurd or the Linux kernel, or it can be used as a standalone operating system distribution for i686, x86_64, ARMv7, AArch64 and POWER9 machines.

In addition to standard package management features, Guix supports transactional upgrades and roll-backs, unprivileged package management, per-user profiles, and garbage collection. When used as a standalone GNU/Linux distribution, Guix offers a declarative, stateless approach to operating system configuration management. Guix is highly customizable and hackable through Guile programming interfaces and extensions to the Scheme language.

by Ludovic Courtès at Tuesday, October 26, 2021

Programming Praxis

Prime Palindromes

Today’s exercise is from a beginner programming web site:

A prime number, such as 127, has no factors other than itself and one. A palindrome, such as 121, is the same number when its digits are reversed. A prime palindrome, such as 131, is both prime and a palindrome. Find the 100th prime palindrome.

Your task is to write a program to find the 100th prime palindrome; answer the question as if you were explaining the task to a beginner programmer. When you are finished, you are welcome to read or run a suggested solution, or to post your own solution or discuss the exercise in the comments below.

by programmingpraxis at Tuesday, October 26, 2021

Mark Damon Hughes

Haunted Dungeon early beta

Hey, it's a new and slightly more usable build (still Mac only) of the Haunted Dungeon! You can now use all the weapons & armor, eat & drink food & potions, might even make it down to floor 2 or 3!

Up in the next few days:

  • Tossing out items. I need to write a new event mode for selecting it, so I didn't feel like it today.
  • Levelling up. Right now you're doomed because you can't heal except by food & potions; or improve, except by very rare (and a long ways down) stat potions.
  • Start getting the actual story into the game. But only the first hints will be in the upper levels, since you can't get far anyway.
  • Main release on Halloween!

Probably next month:

  • Backpack for storing more items. One of the premises of this game is every item's a singleton, there's no stacking. So every item must be useful by itself, and item slot management is hard.
  • Ranged weapons have range.
  • Magic spells.
  • Elemental effects. It's already true that hitting some monsters with sharp or blunt weapons is better, but there's many more interactions when magic gets involved.
  • Much more dungeon dressing, I'm using Vexed's Demonic Dungeon art for that late-'80s, hi-res but low color count effect.

I'd love to get some feedback if it actually works on everyone's machine, because I'm doing some weird tricks to make it launch. As noted on itch, if it doesn't launch, or crashes, try looking at ~/Documents/haunted-dungeon.log, email me with that file.

The game's written in Scheme, running on Chez Scheme, with SDL2 from Thunderchez. Then I have an excessively complex Scheme script that compiles it, and builds a .app structure for Mac, and should also make Linux & Windows builds on those platforms (or in a VM, which is how I do it); been a while since I tried those, but once this is solid I'll include those.

I normally discuss ongoing projects on fediverse, @mdhughes@appdot.net

by mdhughes at Tuesday, October 26, 2021

Sunday, October 24, 2021

Gauche Devlog

Is this an Undefined Behavior?

Is this an Undefined Behavior?

Automated tests of Gauche HEAD on Windows platform started failing since several days ago. The log showed SHA1 digest result didn't match. It's weird, for I haven't touched that part of code for long time.

I isolated the reproducible condition. It happens with the fairly new gcc (11.2.0) with -O2. It doesn't exhibit without optimization, nor with gcc 10.2.0 or other previous versions of gcc I have.

The problematic code is Aaron D. Gifford's SHA implementation sha2.c ( http://www.aarongifford.com/computers/sha.html ). It was last updated in January 2004, so it's pretty old, but I think it's still widely used.

I narrowed down the problem to around here:

        /* Set the bit count: */
#if BYTE_ORDER == LITTLE_ENDIAN
        /* Convert FROM host byte order */
        REVERSE64(context->s1.bitcount,context->s1.bitcount);
#endif
        void *buf56 = &context->s1.buffer[56];
        *(sha_word64*)buf56 = context->s1.bitcount;

        /* Final transform: */
        SHA1_Internal_Transform(context, (sha_word32*)context->s1.buffer);

In our case, BYTE_ORDER is LITTLE_ENDIAN. REVERSE64 is a macro to swap the byte order of a 64bit word. context->s1.bitcount is uint64_t, and context->s1.buffer is an array of unsigned chars. What it does is to store 64bit-word of bitcount into the buffer from 56th octet in the network byte order, and calls SHA1_Internal_Transform.

It compiles to this code with optimization:

   25ca75e9d:   48 8b 53 18             mov    0x18(%rbx),%rdx
   25ca75ea1:   48 0f ca                bswap  %rdx
   25ca75ea4:   48 89 53 18             mov    %rdx,0x18(%rbx)
   25ca75ea8:   48 89 d9                mov    %rbx,%rcx
   25ca75eab:   4c 89 e2                mov    %r12,%rdx
   25ca75eae:   e8 8d fa ff ff          call   25ca75940 <SHA1_Internal_Transform>

Here, %rbx contains the pointer to context, and %r12 to context->s1.buffer. The first three instructions swap the 64bit word. (By the way, REVERSE64 macro is written with shifts and bitmasks. Gcc cleverly figures out its intent and replaces the whole expression by a bswap instrucion.) The next three instruction is the calling sequence of SHA1_Internal_Transform.

Right. There're no instructions emitted to store bitcount into *buf56. I checked the assembly after this but there're no instructions for the store either.

If I insert a dummy external function call before SHA1_Internal_Transform like this:

        /* Set the bit count: */
#if BYTE_ORDER == LITTLE_ENDIAN
        /* Convert FROM host byte order */
        REVERSE64(context->s1.bitcount,context->s1.bitcount);
#endif
        void *buf56 = &context->s1.buffer[56];
        *(sha_word64*)buf56 = context->s1.bitcount;

        puts("foo");

        /* Final transform: */
        SHA1_Internal_Transform(context, (sha_word32*)context->s1.buffer);

Then the storing to *buf56 appears (mov %rdx, 0x58(%rbx)):

   25ca75e9d:   48 8b 53 18             mov    0x18(%rbx),%rdx
   25ca75ea1:   48 0f ca                bswap  %rdx
   25ca75ea4:   48 89 53 18             mov    %rdx,0x18(%rbx)
   25ca75ea8:   48 8d 0d 8f c7 00 00    lea    0xc78f(%rip),%rcx        # 25ca8263e <.rdata+0x9e>
   25ca75eaf:   48 89 53 58             mov    %rdx,0x58(%rbx)
   25ca75eb3:   e8 60 2e 00 00          call   25ca78d18 <puts>
   25ca75eb8:   4c 89 e2                mov    %r12,%rdx
   25ca75ebb:   48 89 d9                mov    %rbx,%rcx
   25ca75ebe:   e8 7d fa ff ff          call   25ca75940 <SHA1_Internal_Transform>

Now, accessing type panned pointer can break the strict aliasing rule. The gcc might have figured the storing into *buf56 had nothing to do with SHA1_Internal_Transform. But I feel there still needs to be a leap that it completely eliminates the store instruction.

Does *(sha_word64*)buf56 = context->s1.bitcount triggers Undefined Behavior? That's why gcc is entitled to remove that code?

Tags: gcc, Optimization, UndefinedBehavior

Sunday, October 24, 2021

Thursday, October 21, 2021

Gauche Devlog

Better test failure report

Better test failure report

I keep Gauche's test framework ref:gauche.test intentionally simple--a test evaluates a given expression and compares its result with the expected result; if they don't agree, reports it. That's all.

It doesn't have fancy knobs and dials, but it does the job. Fancy features can be written using Gauche's other features; e.g. if you need setup/teardown, you can just wrap tests with unwind-protect. I prefer this kind of explicit code to fat frameworks in which you need to track down its documents and (sometimes) implementation to know what exactly is done.

However, there has been one frustration: I can't easily change how the test failure is reported. Especially, when a test yields a large amount of results and it doesn't agree with expected one, it is hard to tell where is the difference, by looking at the entire expected and actual results.

Now I can have it. See the following test:

(test* "Beatrice"
       ;; expected
        '("What fire is in mine ears?  Can this be true?"
          "Stand I condemned for pride and scorn so much?"
          "Contempt, farewell, and maiden pride, adieu!"
          "No glory lives behind the back of such.")
       ;; actual
       "What fire is in mine ears?  Can this be true?\n\
        Stand I condemn'd for pride and scorn so much?\n\
        Contempt, farewell! and maiden pride, adieu!\n\
        No glory lives behind the back of such.\n"
        test-check-diff           ; check
        test-report-failure-diff) ; report

The expected text and the actual text have slight difference. This reports the difference in unified diff format.

ERROR: GOT diffs:
--- expected
+++ actual
@@ -1,4 +1,4 @@
 What fire is in mine ears?  Can this be true?
-Stand I condemned for pride and scorn so much?
-Contempt, farewell, and maiden pride, adieu!
+Stand I condemn'd for pride and scorn so much?
+Contempt, farewell! and maiden pride, adieu!
 No glory lives behind the back of such.

The third argument of test* is to compare the expected and actual result. If you prepare expected text in one big string, you can just use the default one; test-check-diff adds a bit of convenience by accepting a few different formats.

The fourth argument is the main addition. It accepts a report proceudre which is called when the expected result and the actual result didn't match, with three arguments, **message**, **expected-result** and **acutual-result**. The **message** argument is the first argument passed to test*.

The test-report-failure-diff uses text.diff module to display the difference of the results in diff format (ref:text.diff).

You can customize reporting as you wish. Another custom reporting we'd like to have is to show difference of tree structures.

Please refer to the manual for the details. (Before releasing 0.9.11, you can view the draft document.

Tags: 0.9.11, Testing, text.diff

Thursday, October 21, 2021