Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lambda expressions with guards and multiple clauses (was: \ of, -XMultiWayLambda) #302

Merged
merged 46 commits into from Sep 21, 2021

Conversation

JakobBruenker
Copy link
Contributor

@JakobBruenker JakobBruenker commented Nov 29, 2019

The proposal has been accepted; the following discussion is mostly of historic interest.


implemented in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/7873


This proposal introduces a new extension -XMultiWayLambda that allows a new expression introduced by \ of, similar to lambdas but with guards and multiple clauses, enabled by implicit layout.

Rendered

@Ericson2314
Copy link
Contributor

+1. I don't really like guards, but as long as we have guards it's just inconsistent to not allow them here.

@simonpj
Copy link
Contributor

simonpj commented Nov 29, 2019

There's something funny about this. We already have LambdaCase which allows

   \case { Just x | x > 3 -> True
                  | otherwise -> False
         ; Nothing -> True }

This alllows multiple alterantive patterns and guards, but I think it is limited to one argument.

You propose that LambdaGuards should allow

  \p (Just x) q | x > 3 -> p
                 | otherise -> q

which allows multiple arguments, and guards, but only one set of pattern matches.

Ordinary function definitions allow multiple arguments, multiple alternative patterns, and guards.

There is clearly someting a bit unsatisfying about this state of affairs. I wonder about instead generalisiing \case to multiple arguments, so you could write

   \case { p (Just x) q | x > 3 -> p
                        | otherwise -> q
          ; p Nothing _ -> p }

@JakobBruenker
Copy link
Contributor Author

@simonpj Extending -XLambdaCase to multiple arguments certainly seems like it has some potential, and I can add it to the alternatives section when I get a chance. However, there's a few reasons why I don't think it should supersede this proposal:

  • Even if we extend the functionality of -XLambdaCase, it still seems inconsistent to not allow guards in lambda expressions directly.
  • Extending -XLambdaCase in this way makes it inconsistent with how regular case expressions work, since they only allow a single term to be examined. One might consider extending the regular case expression as well, however, that's tricky: case a b of is already interpreted as function application of a to b
  • this would be a breaking change: \case Just x -> x is currently parsed as a single pattern, but presumably would be parsed as two patterns with this change - the alternative is to have some different syntax, for which no obvious and elegant solution occurs to me.

@JakobBruenker
Copy link
Contributor Author

JakobBruenker commented Nov 29, 2019

That said, your comparison to ordinary function definitions does make me wonder whether there's a nice way to extend regular lambda syntax (without case) to multiple sets of pattern matches.

The following wouldn't work:

foo
  \(Just x) n -> Just n
  \Nothing _ -> Nothing

Since it's currently interpreted as two lambdas being passed to foo, when-XBlockArguments is enabled.

EDIT: This is incorrect - I had been under this impression because I had been doing something similar with \case, but regular lambda expressions (at the moment) aren't layout heralds. It's interpreted as foo (\(Just x) n -> Just n (\Nothing _ -> Nothing)).

I'm just spitballing here, but maybe it could work if \ introduced a new layout? This could potentially allow something like

foo
  \(Just x) n -> Just n
   Nothing _ -> Nothing

This would achieve parity between lambda expressions and ordinary function definitions (when combined with this proposal), but I don't know if this is the best way to accomplish that.

I'd love to hear what everyone's thoughts about it are though.

@nomeata
Copy link
Contributor

nomeata commented Nov 29, 2019

This seems to reiterate the same discussions we had when introducing LambdaCase in the first place. Here are some random links that might shed some light on that:
https://gitlab.haskell.org/ghc/ghc/issues/4359#note_51122 (and the other comments there, of course)
https://gitlab.haskell.org/ghc/ghc/wikis/lambdas-vs-pattern-matching

@JakobBruenker
Copy link
Contributor Author

JakobBruenker commented Nov 29, 2019

Thank you for bringing up those links, @nomeata. Looks like I'm certainly not the first to think about that syntax for multi-clause lambdas, and it has a few issues.

The to me most compelling alternative raised in the thread is to use a keyword for multi-clause lambdas, though this comes with problems of its own, most prominently which keyword to use. It would be great if we could reach consensus on a design in context of this discussion, but if not I don't think it should significantly hold up the proposal as it stands, since it has value on its own and should be, I believe, much less controversial.

Edit: though in light of this I'll have to think more about the fact that lambdas aren't layout heralds, whereas multi-way ifs are. This could lead to some annoyances in the proposal as is.

@Ericson2314
Copy link
Contributor

Ah if this requires layout too, I night rather just have a -XLambaLayout that changes everything at once.

Besides those links above, soem of the first ghc-proposals in this repo were about these things, and my recollection was that a layout-based lambda would allow everything missing that's wanted today. The only downside was the inability to write things which the community increasingly considers bad style.

@JakobBruenker
Copy link
Contributor Author

JakobBruenker commented Nov 29, 2019

@Ericson2314

Ah if this requires layout too, I night rather just have a -XLambaLayout that changes everything at once.

I've thought about it a bit more, and, while I'm not very familiar with layout rules, I actually think it shouldn't be much of an issue, with the current proposal. With just that, lambdas would simply continue not heralding layouts, and only a guard in a lambda would herald a new layout.

That said, in principle I'm not opposed to making a broader change and using a different extension name for it.

Edit: Ah, thought about it some more again and actually it may not be that easy. The problem (if I'm understanding layouts correctly) is that it's not clear which token should be the layout herald. It can't be the | because that has to be the first token after the herald, and the tokens before that are parts of patterns, which occur regardless of whether there are guards. This could actually be a big problem with the proposal as it currently stands, though I might be missing an obvious solution. I'll take a look at the code of where guards are currently implemented to learn more.

my recollection was that a layout-based lambda would allow everything missing that's wanted today. The only downside was the inability to write things which the community increasingly considers bad style

You're probably referring to this proposal: #18

The most common complaint appears to that if lambdas being layout heralds were implemented under an extension, old style monadic style wouldn't work quite as it did, so this wouldn't work:

do
  f a >>= \b ->
  g b >>= \c ->
  h c

I believe even indenting it like this wouldn't work:

do
  f a >>= \b ->
    g b >>= \c ->
      h c

One option would be something like

do
  f a >>=
  \b -> g b >>=
    \c -> h c

Or of course it could be simply replaced with do notation, which is probably the sensible thing to do anyway:

do
  b <- f a
  c <- g b
  h c

Though one commenter also mentioned that the following wouldn't work:

foo = \x -> do
  a x
  b

This would have to be replaced by one of the following options:

foo =
  \x -> do
    a x
    b

foo' = \
  x -> do
  a x
  b

or something similar.

So the question is, are we okay with these examples not working with the respective extension enabled?

@JakobBruenker
Copy link
Contributor Author

(Closing was a misclick, sorry)

@JakobBruenker JakobBruenker reopened this Nov 29, 2019
@Ericson2314
Copy link
Contributor

Ericson2314 commented Nov 30, 2019

Hmm, that is worse than I thought. Thanks for writing those examples.

@phadej
Copy link
Contributor

phadej commented Nov 30, 2019

There is abandoned proposal about multi-argument \case and a like #18

@Ericson2314
Copy link
Contributor

Ah, I can't tell if I'm being sarcastic or not, but we could do it for explicit braces only and not layout, until the layout is figured out.

I want to add a layout rule where one layout herald can be "dominated" by another, e.g.

do x do
  y
  z

But I don't know the algorithm well enough to make a concrete suggestion.

@JakobBruenker
Copy link
Contributor Author

JakobBruenker commented Nov 30, 2019

Another option would be to only have \ as layout herald (and only allow multiple clauses) if it's immediately followed by a newline. This seems to be a parse error currently, so I think it'd actually be backward compatible.

Edit: Correction, it is not a parse error currently, so I'll have to think a bit about how it would change current behavior.

@JakobBruenker
Copy link
Contributor Author

Regarding layout for guards:

For most places where guards can occur, they do not begin a layout, so the following works:

a = case () of _ | True -> 0
                | False -> 1

b = let foobar | True = 0
          | False = 1
    in foobar

With MultiWayIf, initially it wasn't a layout herald, but the following example led to it being changed:

x = if | False -> if | False -> 1
                     | False -> 2
       | True -> 3

It would be intuitive for x to be 3, but without layout, it failed. I believe the above guard examples don't have this issue since they are inside a larger layout thanks to case or let.

However, this does make things slightly inconsistent, since for example the following does not parse (compare with the first example):

a' = if | True -> 0
       | False -> 1

Lambdas (currently) are not layout heralds, so we could potentially run into the same issue as MultiWayIf had.

I believe that the lexer can be adjusted such that patterns in lambdas are layout heralds iff followed by a |. If this is possible I think it would solve the layout issue for lambdas in guards nicely.

Optionally, to make it more consistent, this same rule could be applied to guards elsewhere if the extension is enabled. (Or similar rules if guards are preceded by an identifier instead of a pattern).

@JakobBruenker
Copy link
Contributor Author

JakobBruenker commented Dec 3, 2019

Well, I feel like a fairly coherent idea is forming here.

  1. Lambdas can have guards, and guards introduce layout in the same way as in MultiWayIf
  2. Lambdas can have multiple clauses iff a newline immediately follows the \
    • 1. and 2. can make \case obsolete since they are strictly more powerful (except that you need a newline after the \, but not after \case
  3. Zero argument lambdas are possible (e.g. \ -> a + b)
    • 1. and 3. can make MultiWayIf obsolete since if | ga -> a | gb -> b can be written as \ | ga -> a | gb -> b
    • With 1., 2., and 3., one could even use lambdas instead of regular function definition, i.e. instead of
f A S | g1 = a
      | g2 = b
f _ _ | g3 = c
      | g4 = d

one could always write

f = \
  A S | g1 -> a
      | g2 -> b
  _ _ | g3 -> c
      | g4 -> d

This is (more or less) something @Ericson2314 suggested in #18 (comment), though he mentions "weird type inference issues", which might be worth looking into.

These three points give lambdas parity with regular function definition and make the language simpler in the sense that the functionality of three (or four) features has been combined into one, in a way that's consistent with the existing language.

I'll write this into the proposal later today.

@goldfirere
Copy link
Contributor

I had been dubious of this line of inquiry. No longer!

One possible improvement (or unnecessary complication, depending on your point of view): where clauses. Right now, we have

foo (Just x)
  | x < 0 = ...
  | let y = blah + 1 = ...
  where
    blah = x + 5

Note that the x is in scope in the where clause and that the where-bound blah is in scope in the guards. This is very useful. However, a where clause cannot scope over multiple top-level equations, which is sometimes annoying. (Obviously, we can't have it all: a where clause over multiple top-level equations cannot also have variables bound in those equations in scope.)

With the new syntax, could we have

foo = \
  (Just x) | x < 0 -> ...
               | let y = blah + 1 -> ...
    where blah = x + magicNumber
  Nothing -> magicNumber
  where
    magicNumber = 5

Note that my first where is indented w.r.t. the (Just x). It has x in scope; anything bound in this where is in scope over all the guards. My second where is at the same level of indentation as the (Just x). It does not have x in scope, but its definitions are in scope over all the alternatives. This where could be out-dented from the (Just x), too.

One might reasonably argue that this indentation-awareness is too subtle. I wouldn't argue hard against. But it seems like a nice opportunity with this new syntax. I really like it.

@akhra
Copy link

akhra commented Dec 3, 2019

We already have nested wheres, I don't think indentation sensitivity here is going to throw anyone for a fresh loop.

And that's good, because I love this.

@JakobBruenker
Copy link
Contributor Author

JakobBruenker commented Dec 3, 2019

@goldfirere I like that idea, I'll add it to the proposal. I was going to say that for consistency's sake, it would make sense to add that same where behavior to clauses of case ... of, but it actually looks like that's already possible, which is another point in favor of this where syntax.

@goldfirere
Copy link
Contributor

goldfirere commented Dec 3, 2019

Not to lessen our enthusiasm, but this proposal does have a negative: it introduces the first place where Haskell is newline-aware. That is, with this proposal, a newline has different semantics than other whitespace. This is a new aspect of Haskell (outside of string/quasi-quote literals). Of course, newlines have interacted with indentation for eons, but these have always been equivalent:

tok1 tok2 tok3

and

tok1 tok2
          tok3

With your proposal, that's no longer true. To me, this isn't a significant downside, but it's making the language even more whitespace-aware than previously. This relates, of course, to the fact that it's not backward compatible.

\
  x
  y -> blah

was previously acceptable, but will no longer be with this proposal (IIUC).

@goldfirere
Copy link
Contributor

One more thing: might I humbly suggest -XLambdaLayout. It's not about guards any more!

@JakobBruenker
Copy link
Contributor Author

JakobBruenker commented Dec 3, 2019

@goldfirere Yes, I've looked into that, and only three packages on hackage currently have a newline immediately after a backslash in a lambda, and I believe none of them would be broken by enabling the extension (see edit). Still, I certainly would prefer if newlines didn't have to matter.

With the most straightforward specification I agree that your example wouldn't work.

I was considering -XExtendedLambdas, but I like -XLambdaLayout (incidentally also what @Ericson2314 suggested further up in the thread).

Edit: Ah, actually I was slightly wrong about newlines after \ in lambdas on hackage:

  • package KiCS-debugger has 1 use, of which 1 would be broken (iff the extension were enabled)
  • package saltine has 15 uses, of which 1 would be broken
  • package radixtree has 2 uses, of which 2 would be broken

Still, not too bad I think, especially since this incompatibility is guarded behind an extension and it would be easy to adapt code.

@goldfirere
Copy link
Contributor

My worry isn't really about back-compat -- it's more about humans having to reprogram their brains' internal Haskell lexers. Again, this isn't a deal-breaker for me. It's just (to me) a drawback. The benefits outweigh the drawback.

Sorry to shamelessly repeat @Ericson2314's suggestion instead of looking back. -XExtendedLambdas is fine with me, too. My post was more about not using -XLambdaGuards.

@Ericson2314
Copy link
Contributor

Ericson2314 commented Dec 3, 2019

Here's a question, if we make \ always layout with the extension now, is it backwards compatible to add the newline trick later?

I would rather not due newline sensitivity until later when we've exhausted other options, and per #302 (comment) I think it's a deficiency with layout in general, and not lambda in particular.

@JakobBruenker
Copy link
Contributor Author

@Ericson2314

is it backwards compatible to add the newline trick later?

In other words, is there an example that compiles if \ heralds a layout (w/o newline), but doesn't compile/means something else if it doesn't herald?

Yes, if you use multiple clauses:

\(Just x) -> 0
 Nothing -> 1

So no, adding the newline trick later would not be backwards compatible, unfortunately.

@eschnett
Copy link

eschnett commented Dec 3, 2019

If -XLambdaLayout is active, then one could use the same rules for \ as for do to simplify things:

  \A -> ...
   B -> ...

The rule is: All lambda clauses must start on the same column. Newlines don't matter. That's equivalent to the choice that all statements in a do block must start at the same column.

@simonpj
Copy link
Contributor

simonpj commented Aug 9, 2021

Guess that is the last question to be decided before we can merge this?

Yes -- and I'm keen to get this resolved. Here is the choice we are discussing:

  1. Add new extension LambdaCases, which implies the existing LambdaCase
  2. Extend the existing LambdaCase to support \cases.

From what I see above I see

Any other opinions, from anyone? Preferably with brief reasoning. Please say within the next week, then I'll get the committee to decide. It's not a big deal, happily.

@int-index
Copy link
Contributor

Better to keep extensions immutable. Introduce LambdaCases and make it imply LambdaCase.

@phadej
Copy link
Contributor

phadej commented Aug 9, 2021

I think the GHC committee should write down a clear guideline document on the "immutability of extensions".

  • Long ago TypeFamilies was extended with closed TypeFamilies (having ClosedTypeFamilies would been possible).
  • ImpredicativeTypes have changed with quick look (though one can argue that it didn't really exist before - though committee could clarify how to add clearly experimental features into GHC - or is it anymore possible at all?)
  • ApplicativeDo grew in power slightly
-- This works with GHC-8.4 but not with GHC-8.2 or GHC-8.0
-- is accepting this a new feature or a bug fix?
{-# LANGUAGE ApplicativeDo #-}
bar :: Applicative f => f Bool
bar = do
    pure True
    pure False
  • LinearTypes could have changed in between but GHC-9.0 was delayed so the changes went in directly. Though here one can also ask whether LinearTypes was considered to be experimental feature (or is it still experimental?) and whether users should expect changes. (I expect more development, IIRC the feature is far from complete and was put out to gather feedback).

  • If TemplateHaskell gets improvements (e.g. the typed variant), some changes may be considered as bug fixes, but some (like Quoted instances as described in staged-sop paper) are features which may as well be considered an extension on its own. (In fact TypedTemplateHaskell could been a separate extension).

Others and I have brought up the problem of extension versioning previously. It would be very nice to have a written document on what to expect. Being consistent (from now on) would be very welcome. (My current expectation is that anything can change at any point, and I use my experience to judge what is likely to change and what isn't. And I have to admit I didn't think the current LambdaCase is the end of story nor this proposal).

In the light of previous, and IMHO, simply extending LambdaCase is won't be bad (especially as new extension is just one letter off, so e.g. tab-completion in not-so-smart editors will pick the less powerful one as it's shorter). I learned to always be explicit about GHC versions in my developments, and not assume that GHC-next won't break any code (I have plenty, something always breaks).

@JakobBruenker
Copy link
Contributor Author

JakobBruenker commented Aug 10, 2021

how to add clearly experimental features into GHC - or is it anymore possible at all?

-XOverloadedRecordUpdate was added recently and is experimental, and the User guide states that it is intended to be changed in the future:

EXPERIMENTAL
This design of this extension may well change in the future.
[...]
We anticipate this restriction to be lifted in a future release of GHC with builtin support for setField.

(though I suppose it's possible that adding setField to base doesn't actually require changing the extension itself)

@simonpj
Copy link
Contributor

simonpj commented Aug 11, 2021

I think the GHC committee should write down a clear guideline document on the "immutability of extensions".

Fair enough -- why not write to ghc-steering-committee@haskell.org? To me it seems that the tension is:

  • To maximise back-compat, always make new extensions, never deprecate
  • To maximise the coherence of the final language, rationalise extensions to a form that "makes sense" without knowing the history.

I incline to the latter, hence suggesting extending LambdaCase rather than adding another extension. (Thought experiment: if we introduced both \case and \cases simultaneously, would we have defined two extensions? No.)

Returning to the (1) vs (2) choice above, does anyone else want to express a view? It sounds as if @phadej leans towareds (2)

@Ericson2314
Copy link
Contributor

If we versioned language extensions --- not such that code could do {-# LANGUAGE Foo 5 #-} that would be too many knobs, but such that Cabal files could specify their language feature deps more reliably --- we might be able to have our cake and eat it too.

@tomjaguarpaw
Copy link
Contributor

People from outside the community have the (incorrect) impression that language extensions mean that Haskell is in fact several incompatible languages. I don't believe that the incorrect impressions of outsiders should drive our decision making but in this case I really dread the consequences of giving the impression that Haskell is several numbered incompatible languages.

@Ericson2314
Copy link
Contributor

That is fair. One can similarly argue that listing extensions wastes effort on the fiction that there are non-GHC (-based) compilers for Haskell people actually use :/.

@cgibbard
Copy link
Contributor

People from outside the community have the (incorrect) impression that language extensions mean that Haskell is in fact several incompatible languages.

When the truth of the matter is that it's one, increasingly complicated language.

@ivanperez-keera
Copy link

ivanperez-keera commented Aug 13, 2021

To me it seems that the tension is:

  • To maximise back-compat, always make new extensions, never deprecate
  • To maximise the coherence of the final language, rationalise extensions to a form that "makes sense" without knowing the history.

A language that maximizes back compat will grow indefinitely. Haskell already grows very fast (too fast for my taste). That just results in a language that is too big to be understandable, filled with too many language constructs or extensions whose combined effect is too hard to predict.

A language that maximizes coherence reduces entropy, at the expense of breaking language compatibility. That's why it is important that such changes are rare, coordinated, and deployed in a way that gives people a lot of time and help to adapt. The good consequence is that, when people feel they are being given importance just by being told how the plan affects them and by receiving help to adapt their code, they will feel more included, resulting in an overall more cohesive community to which they will feel compelled to give back.

In economic terms, (1) has lower short-term cost but a higher long-term cost, (2) has a higher short term cost but a lower long term cost.

However you look at it, if we can afford it, I believe (2) is a better path.

@nomeata
Copy link
Contributor

nomeata commented Aug 13, 2021

May I suggest that the general discussion, that is no longer tied to \cases, happens at #432 or a new thread there?

@JakobBruenker
Copy link
Contributor Author

@simonpj Since it's been a few weeks, is this ready to go to the committee again to decide on whether to use -XLambdaCase or a new extension?

@simonmar
Copy link

It's a difficult call and I have sympathy with all the points raised so far on both sides. But let me argue in favour of (1):

  • I see extensions themselves as a temporary measure. The ultimate goal is convergence, where extensions are rolled into GHC20xx and then into Haskell20xx. (with a few exceptions). Nobody actually wants extension flags (again, with a few exceptions) - they're a necessary evil brought on by the desire to evolve the language while people are actually trying to use it.
  • The main purpose of extension flags is backwards compatibility during this (long) migration.
  • Given that, I think it makes sense to prioritise backwards compatibility over coherence in the current set of extensions.

@simonpj
Copy link
Contributor

simonpj commented Sep 15, 2021

@JakobBruenker I am sorry this has taken so long. I have been distracted. I will get it decided asap.

@simonpj
Copy link
Contributor

simonpj commented Sep 21, 2021

I took a vote of the steering committee of (1) vs (2) and ended up with (2).

So @JakobBruenker could you modify the proposal to say that we'll simply extend the existing LambdaCase extension to support \cases, rather than add a new flag?

Then we can merge. Thanks!

@JakobBruenker
Copy link
Contributor Author

@simonpj @nomeata Thanks! The proposal already talks about extending -XLambdaCase, so sounds like it can be merged

@simonpj
Copy link
Contributor

simonpj commented Sep 21, 2021

OK @nomeata could you pull the trigger please?

@nomeata nomeata merged commit ab97c78 into ghc-proposals:master Sep 21, 2021
@nomeata nomeata added Accepted The committee has decided to accept the proposal and removed Pending committee review The committee needs to evaluate the proposal and make a decision labels Sep 21, 2021
@nomeata
Copy link
Contributor

nomeata commented Sep 21, 2021

Trigger pulled. The conversion from .md to .rst was not flawless, but mostly in the Alternative section. I hope that’s ok

@JakobBruenker
Copy link
Contributor Author

@nomeata Looks like the tables have a few instances of the string :raw-latex: that should be removed. Sorry to put you through the ordeal of having to convert raw html, I've since started using .rst in other proposals.

@nomeata
Copy link
Contributor

nomeata commented Sep 21, 2021

@nomeata
Copy link
Contributor

nomeata commented Apr 5, 2022

Implemented in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/7873!

@nomeata nomeata added the Implemented The proposal has been implemented and has hit GHC master label Apr 5, 2022
@Ericson2314
Copy link
Contributor

Thanks so much @JakobBruenker for pushing this through!

@tomjaguarpaw
Copy link
Contributor

Can someone with admin powers edit the initial post to link to the implementation: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/7873

@JakobBruenker
Copy link
Contributor Author

@tomjaguarpaw done

@phadej phadej mentioned this pull request Nov 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Accepted The committee has decided to accept the proposal Implemented The proposal has been implemented and has hit GHC master
Development

Successfully merging this pull request may close these issues.

None yet