I seem to have gotten a very basic qualified type thing working. still need to noodle with it more, though.
Qualified types introduce a type predicate that needs to be checked after inferring each expression. For example, let's say the signature of '+' is 'a, a -> a' where 'a' is any type. Addition only makes sense for certain types of data, so we can then specify which types are valid with a predicate. For example: where 'a' is a float or an int. Adding two booleans is valid as far as the function signature is concerned, but it will then fail the predicate check.
One limitation of this approach seems to be (meaning I haven't found a paper/article stating otherwise yet) that it cannot express all of the possibilities of function overloading. For example, in GLSL it is valid to multiply a matrix by a vector. It is also valid to multiply a vector by a matrix. Both forms return a matrix, but the signatures are different. The first is 'a, b -> a' and the second is 'b, a -> a'. This would require two differently named functions in a qualified type system.
What might be useful is allowing for a signature like 'a, b -> c' and associate substitutions with predicate conditions. If 'a' is a vector and 'b' is a matrix then substitute 'b' for 'c'. If 'a' is a matrix and 'b' is a vector, substitute 'a' for 'c'.
Okay so this strategy seems to be working! Hopefully will have some cool code to share soon.
Here's a simple example program:
(top-level ((in int x))
(let ((f (lambda (x) (+ x 1))))
(f 2)))
The only type declaration is on the shader input attribute 'x'. Here's the typed program that the compiler generates:
(t ((primitive int))
(top-level
((in int V0)
(function
V2
(t ((for-all
((tvar T3) (tvar T7))
(-> ((tvar T3)) ((tvar T7)))
(and (= (tvar T3) (primitive int))
(substitute (tvar T7) (tvar T3)))))
(lambda (V1)
(t ((tvar T7))
(primcall
+
(t ((tvar T3)) V1)
(t ((primitive int)) 1)))))))
(t ((primitive int))
(call (t ((-> ((primitive int)) ((primitive int)))) V2)
(t ((primitive int)) 2)))))
The interesting bit is the 'for-all' expression that generalizes the function 'f' to accept any arguments as long as they satisfy the predicate (the 'and' expression). In this case the only valid type for 'x' is an integer.
The compiler is currently unable to emit GLSL code for the function because it doesn't yet know how to translate a 'for-all' into a series of functions for each valid type combination. That's the next step.
Okay now I can emit GLSL for functions with multiple signatures!
Contrived input program:
(let* ((f (lambda (x y) (+ x y)))
(z (f 1.0 2.0)))
(f 3 4))
'z' is a useless binding that just demonstrates that 'f' can be called with floats and ints. good thing I don't do dead code elimination yet.
GLSL output:
void V2(in int V0, in int V1, out int V4) {
int V5 = V0 + V1;
V4 = V5;
}
void V2(in float V0, in float V1, out float V6) {
float V7 = V0 + V1;
V6 = V7;
}
void main() {
float V8 = 1.0;
float V9 = 2.0;
float V10;
V2(V8, V9, V10);
float V3 = V10;
int V11 = 3;
int V12 = 4;
int V13;
V2(V11, V12, V13);
int V14 = V13;
}
I now have basic struct types working. The qualified type logic was extended to support them and overloaded function generation was extended to deal with structs as arguments. I also added function signatures for a number of built-in GLSL functions. I can once again compile the simple vertex shader I use for 2d sprites:
(top-level
((in vec2 position)
(in vec2 texcoords)
(out vec2 frag-tex)
(uniform mat4 mvp))
(outputs
(vertex:position (* mvp
(vec4 (-> position x)
(-> position y)
0.0 1.0)))
(frag-tex texcoords)))
GLSL output:
in vec2 V0;
in vec2 V1;
out vec2 V2;
uniform mat4 V3;
void main() {
float V4 = V0.x;
float V5 = V0.y;
float V6 = 0.0;
float V7 = 1.0;
vec4 V8 = vec4(V4, V5, V6, V7);
vec4 V9 = V3 * V8;
gl_Position = V9;
V2 = V1;
}
I took a look at the type inference code in guile-prescheme and I'm impressed with how elegant it is. However, it's an imperative implementation. My algorithm is functional and I'd like to keep it that way. Once I have something that produces correct output for all of my existing shaders I will try to refactor the code into something nicer.
The next big step was hacking this compiler into my existing shader code so I could actually use the shaders and test that they really work. This required a linking pass that does a bunch of renaming on the separately compiled vertex and shader stages to ensure that vertex output names match up with fragment input names (GLSL needs the names to be the same, ugh!) and that uniform variables between both stages use unique names because they share a global namespace.
ahhh I got my single sprite and batch sprite shaders ported over and *they work*!!!
here's a more complicated shader that I ported to seagull. I use this for stretchable dialog boxes. https://gist.github.com/davexunit/5d9c4f707c349e3a5aada5dae1ce674d
My most complicated shader port yet: the stroke and fill shaders for my vector graphics renderer. https://gist.github.com/davexunit/83e1ed3d71ea11f48e30fc8cc0d568ec
Seagull is feeling pleasantly Schemey! There are two big missing features in the language: Loop syntax (GLSL *severely* restricts looping and recursion is *not* allowed so a loop form is essential, unfortunately) and user defined structs. I already have define-shader-type syntax that I've been using with my existing GLSL shader code so I just need to a way to import those into Seagull code and have the compiler generate the GLSL struct declaration.
Both of those language features will be necessary to port my most complicated shaders, which are the phong and physically-based lighting shaders.
I didn't add either of those things, but I did add a dead code elimination pass to prune unused variable bindings.
okay so I'm going to add syntax for the *very* restricted loops that GLSL allows. GLSL is limited to 'for' loops where the loop variable is initialized to a constant value, incremented by a constant value, compared to a constant value, and actually terminates (you can't say `i = 0; i < 5, i--`.) I intend to not emit 'for' loops at all, opting instead to unroll the loops in a compiler pass. for this to work I'll need a teeny interpreter that figures out all the values of the iterator variable.
since Seagull is purely functional, the loop syntax needs to support a way to accumulate a result. the Schemiest syntax I've come up with is a riff on Scheme's 'do':
(for ((i 0) (< i 5) (+ i 1)) ; iterator
((x 0)) ; accumulators
(+ x i)) ; body
so I implemented this and used the exact code above to test. when I finally got the bugs out I didn't see an unrolled loop at all because the partial evaluator simplified the whole thing to just '10' lol
oops turns out only GL ES has these strict loop limitations. regular GL is a lot more flexible, including being able to express non-terminating loops! didn't read the spec close enough. not sure if I want to do loop unrolling at all, now.
Not much progress on loops but I did do some significant refactors that make it a lot easier to add new primitives and added some REPL commands to make running the compiler more convenient while developing.
@dthompson Yeah, this is what I'd try to do.
@zenhack cool, thanks! and thanks for pointing me towards qualified types.
@dthompson You're welcome!
@dthompson Now I want code blocks with syntax highlighting in Mastodon.
@acousticmirror me too!
@dthompson You could take a page from SERIES instead of LOOP. Too complex?
@dl yeah I find those looping macros too complex for this specific case. they're very general with more complicated syntax.
@dthompson Are you allowed multiple accumulators? If yes, how do you set their values through a loop?
I personally dislike the ((i 0) (< i 5) (+ i 1)), because it's too general. If I'm not allowed to do (for ((i 1) (= (% i 31) 5) (* i 3))) then the syntax should try to not look like I can.
I prefer something like (for (i from 0 to 5)) and (for (i from 0 to 5 by 1)) as explicit syntaxes. They're clear about what the form can do, and they're easy to match in a scheme macro.
@carlozancanaro I plan to allow compound expressions like that if they can be eval'd at compile time.
@carlozancanaro maybe I'll come to prefer the form you suggest over time. I don't hate the syntax I have now so I'm going to keep moving on so I can write some real shader code and see how I like it.
@dthompson@toot.catAnother unsolicited syntax suggestion 
: have a look at @racketlang 's for/fold and similar macros. I find them much simpler than the embed-modula2-for-loops style macros (to be fair, I mainly know / fear the common lisp loop DSL).
@carlozancanaro
@bremner haha I looked the racket docs and I basically implemented a slightly different for/fold! Racket reverses the order for the iterator and accumulator expressions.
@dthompson The expressions I wrote are fundamentally not allowed, though. Using * and % are illegal in the target language, so using them will never be permitted.
I'm still interested in the multiple accumulators question, too. What would it look like to accumulate three values, for instance? Is that allowed?
@carlozancanaro those expressions are allowed in Seagull because the partial evaluator can simplify the expressions down to constants and will unroll the loop. none of those expressions will be emitted to GLSL code and there will be no GLSL 'for' loop at all. there can be only one iterator variable ('i' in my example) but there can be many accumulators. a silly example of this (x and y are accumulators):
(for ((i 0) (< i 5) (+ i 1))
((x 0) (y 0))
(values (+ x i) (- y i)))
@rml@functional.cafe@dthompson @carlozancanaro that's pretty sweet man, a little optimizing glsl compiler
@rml @carlozancanaro haha yeah too bad this particular use-case is turning out to not be what I really want to do. glad to know the constant folding works OK, though!
@dthompson Right, okay. I misunderstood that this isn't actually syntax for a GLSL loop, it's syntax for a compile-time loop in Seagull. That makes more sense of having the more general forms for init/step/condition.
For accumulator syntax, the definition of x is effectively separated into two parts: an initial value, and then in a separate place an iteration form. In a purely functional context there are benefits to keeping them together, with something like:
(for ((i 0) (< i 5) (+ i 1))
((x 0 (+ x i))
(y 0 (- y i))))
Both x and y are bound for the iteration form, but not for the initial value.
I prefer this because it prompts me think of each term as a definition of a recurrence relation (potentially depending on terms of another recurrence relation), rather than a value being updated in a loop. I think this suits the functional model a bit better.
Although now that I type that, one obvious downside is if you want the loop body to be a single function that calculates new values for all the accumulators at the same time. Maybe everything I said in this toot is a bad idea.
@carlozancanaro yeah there are definitely some tradeoffs with that syntax. the scoping is definitely different (and there can be no recursive definitions in glsl). I think I'm heading for a more general loop now that I've realized I was mistaken about something in the GLSL spec.
@dthompson
When I hear things like this I always think of Andy Wingo saying "Damnit GCC" while showing benchmarks of different versions of guile.
@rml@functional.cafe@dthompson I feel like for loops are discouraged in fragment shader code because different compilers handle them differently, so they are very expensive on some GPUs. Its usually better and more common to use feedback buffers in lieu of recursion, which can be accomplished in the vertex shader with the "transform feedback" functions, which also have powerful operators that allow for suspendable threads (in the sense that a pixel is a thread in the glsl model). Perhaps there could be a more natural expression of feedback with macros that compose to expand to a chain of feedback buffers; its usually like bending spoons for beginners, and something that pushes lots of people away when they first have a go at shader programming, but I don't think it has to be like that.
@rml I've never used feedback buffers since it involves bidirectional state and it's not supported on GL 2 but yes, usage of loops should be minimized. I'm just trying to support them in a basic way, at least. I currently use a short for loop in 2 fragment shaders.
@rml after re-reading some of the spec I think that emitting 'while' loops will be the easiest thing. could revisit that if I ever need to target GL ES.
@rml@functional.cafe@dthompson I think im going to get started with vulkan in the coming months, I've said I would for years but never had a real excuse to. Now, looking at the tech crash job market, the only jobs that seem to really looking for ppl in FOSS are vulkan jobs, and considering I've always worked in graphics in c++ im hoping it can be a feasible transition out of the destabilized stage & events industry.
I know it's suppose to be really hard, but I'm hoping that without the pressure to make any artwork it can be fun, and can lead to some interesting experimentation interfacing with scheme.
@rml I'd definitely be interested in whatever you were up to with that!
@rml@functional.cafe@dthompson I guess the general strategy of vulkan is to push as much low level "driver" details to the API as possible in order to minimize demands on compiler authors, which is probably the biggest difficulty to providing performant, portable & maintainable free software graphics software considering there are so many gpu architectures to compile to. Listening to the vulkanize conference from a couple weeks ago, it sounds like the biggest obstacle to vulkan's proliferation is just a lack of people who have suffered through learning it.
But tbh it really does sound like the recently included Virtual Memory Allocator solves much of the major memory management and synchronization issues that formed the bulk of gripes I've read; we'll see, I'm sure its gonna be awful 
I was originally hoping to find a job in functional programming, but I've come full circle to think that, if I can wing it, I'd probably rather do more gritty work that improves the state of Linux graphics, because even if the former is exhausting, at least it actually has a positive impact & helps weaken corporate attempts to monopolize their hardware.
Or I'm just coping as i prepare for a life of toiling in the gnarliest, most incomprehensible bit mines imagineable haha
@rml I like the idea of vulkan but the complexity is intimidating and I'm concerned about performance with the huge increase in ffi calls that would be necessary to use vulkan from scheme. I also don't know what the story is for vulkan on machines that only support GL 2 because I try real hard to make sure my stuff is usable on old machines.
@dthompson Would it be pleasant enough with tail recursion only...?
@alilly GLSL doesn't allow it! I've thought about ways to make Seagull code look like tail recursion that compiles to a for loop but I think it ends up clunky and confusing because there are just so many limitations that apply.