New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make Q (TExp a) into a newtype #195
Conversation
Formatting is funny... fix? Why just
Can you be more explicit? What is "the principle of typed staged programming"? What splices can fail to run? Examples would help a lot.
Can you give examples of what instances you might want to write, and what evolutions you have to go through? |
I don't have a real opinion about untyped template haskell but it seems to me that it's primarily about constructing code fragments explicitly and unsafely using operations from the
The principle is that we should never fail when running a splice. Errors should occur whilst constructing the value that we want to splice in so that we ensure that we can only construct well-scoped and well-typed terms. For a particular situation, imagine if you call
I added this example of a simple DSL where it is useful to write an instance for
|
Seems like there is little interest in this proposal. How long does the committee usually wait before considering them? |
I guess a week is reasonable. Has this proposal been advertised anywhere (mailing lists, reddit), i.e. is there a good chance that people who care about TH are aware of it? But if it is small, and obviously desirable, then a week of silence is reasonable. |
Isn't this going to break pretty much every non-trivial piece of TH in existence? More importantly, it looks like there isn't going to be any easy way to fix that code in a backwards compatible fashion. It would be great if the proposal could talk a bit more about what this will break and what the recommendation will be to unbreak it... In short: the breakage to benefit ratio seems a bit high. |
@harpocrates Firstly the potential breakage is only in programs which use typed template haskell of which there are not that many judging by the very few bug reports and number of quite serious bugs. However, users who use the API safely by quoting and splicing would just have to change their type signatures with this modification. Users who use the operations from the Reflecting on this compared to other changes like, making |
@Ericson2314 what is your objection to this proposal? Will it break the way you use typed template haskell? |
Yes, Also, you propose changing |
What happens with This proposal fixes a mistake in the original design, there is no very straightforward backwards compatibility story precisely because One option for a backwards compatible change (that I am strongly against) is introducing a new module which exposes the new interface and to teach GHC to be able to quote either |
I don't think this is a guiding principle that makes sense to me. The promise is not that we never fail -- it is that we produce well typed code, if we happen to succeed. It is always possible to fail -- e.g., exceptions, nontermination, etc. |
Yes mainly I was confused on what failures there are, but secondarily exactly what @gbaz said. There are issues with |
Gershom, you are right to clarify this and I agree that I assume people write programs that terminate and don't throw exceptions but I don't see how it is related to the content of the proposal? Which is
If you are referring to (2), the point is that if a user throws an exception in their program then they should expect it to fail. However, if they use the currently advertised API by calling
@Ericson2314 What do you mean by "what failures there are"? The issue you mention about |
Yes, the things I brought up indeed are irrelevant here, that was my point! When you write "The principle is that we should never fail when running a splice." I couldn't think of any failures that relate to typed template Haskell in particular. So if like @gbaz I think splicing failure in general has nothing to do with typed TH (the point is to prevent errors after splicing (type errors) not during), and there are no specific splicing failures which are relevant to typed TH, then splicing failure disciplines and the typedness of template Haskell variants are truly orthogonal. |
So to summarise the discussion here. There is one objection of substance which is that of backwards compatibility. However, I would consider it an absolute travesty if the proposal was rejected on these grounds due to the far more invasive breakages which happen every release. Given that no users have come forward to express this concern, I believe that is should also be disregarded. I also grepped my old copy of Hackage (from 2017) for occurences of
|
@mpickering I wonder why it must be a newtype over Indeed, the name
It is obvious how to interpret it in
And this opens up a possibility of writing other interpreters for |
@int-index My primary problem is that you can't write instances for Generating fresh variables is also one of the most dangerous operations! It's very easy to end up with an AST which contains unbound variables. (FWIW, I have also implemented an alternative serialisation mechanism for quotes which serialises core expressions rather than to the template haskell AST, this is sufficient if you don't need to inspect it. This isn't relevant to this proposal though.) |
At least that's one dangerous operation instead of the ability to embed arbitrary Generating fresh variables can be easily done in pure code with the It would help me greatly to reason about what In case the only motivation is merely to write instances, I suggest the name |
On the original proposal: it would help in the cases where you need need an instance for On the "weakened Q" proposal:
It's often useful to mix code generation with other effects --- in particular, effects that can make non-local transformations to code. For example, this is from the abstract of Kameyama, Kiselyov and Shan's (2014) Combinators for impure yet hygienic code generation:
So I'm concerned that the weakened WQ is likely to be a bit too weak in practice. |
You're right that it wouldn't "help" instances for I think the best path to "weakened Q" is to disentangle |
@mpickering If we make quoting and splicing overloaded, as you propose in Trac #16178, would it not subsume this proposal?
Then separately, perhaps in another library,
|
I'm not sure how it would help the |
@int-index I think the point would be to make There would still be the same problems with backwards compatibility with the overloaded proposal as you can't easily write an instance for |
OK. I still have a few gripes about the name
|
Improving support for instances is a good motivation for something. But this "semi-purity" stuff feels weird to me. Another reason is that I think it makes sense to have IO effects inside So having a weakened Q seems entirely orthogonal to the question of having a newtype for the purpose of writing instances, and I think the |
Gershom,I think the intention is that "weakened Q" supports only the operations that are necessary to make the representations of code. This doesn't include doing IO. Then I agree you're right that this is orthogonal to the proposal. |
Is there anything else blocking this proposal? All objections have been dealt with. |
Earlier I wrote
To answer my own question: we also have That is, in the untyped world we construct fragments of many different syntactic categories. But in the typed TH world we can only construct expressions. To me that helps explain why we don't want lots of newtypes for |
@mpickering While you pointed out correctly that Seeing that you yourself proposed |
@Ericson2314 EDIT; TL;DR you might want to have myCode :: ... => Code m a
myCode = joinCode $ do
x <- someSideEffect
return (makeCodeWith x) Simple. |
I am just going to retract my parenthetical, as anything about |
If unsafeTExpCoerce :: m Exp -> m (TExp a) be kept at current type, and rather new unsafeCodeCoerce :: m Exp -> Code m a to be introduced. It's weird to have function with Also note, that we have |
A something to consider. I just run (into my previously mentioned experiment), that instance (Lift a, Num a, Quote m) => Num (Code m a) where
fromInteger x = liftTyped (fromInteger x)
... Would be nice to have. And same for |
My apologies -- it's looking much clearer now, thank you.
|
@Ericson2314 Using effects if a fundamental part of code generation, for example, if you construct a code generator which constructs division, then you probably want to fail at compile time if you would construct a division by zero. Therefore having the monadic context is a big advantage that TTH has over metaocaml and dotty. Another example, I have implemented let insertion using the monadic context ( https://github.com/mpickering/let-insert/blob/master/LetInsert.hs ). Thanks, I was going to let all this stuff shake out in the implementation where it becomes clearer.
I have also started the implementation. |
Thanks Matthew. Can you just add at the beginning of Change Specification that the section gives the complete API for both Also, I was a bit boggled by
We really really shouldn't be exposing
(It's a pity that And there is no reason for Moreover, like
|
Updated the proposal again with your comments. Patch is also available to review: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/3358 It makes no difference other than potentially annoying users to hide The reason |
Bu tit makes a big difference? If I see "unsafe" something I know something is unsafe. If I see I accept that this bug predates this proposal, but the proposal refines the client API of Code and TExp. Let's get it right.
Interesting. The implementation is:
That uses |
Yes, I agree the monad is fundamental! That's why I think it's weird to put it in a newtype. I've never seem anyone newtype The motivation of "just avoiding some wrapping/unwrapping" still seems incredibly weak to me, put forth by those that write the code at the expensive of those that need to read the code later. Imagine if you need to read your coworker's or classmate's TTH code, and are new to this stuff yourself. Would you rather read something with plain old wrapping/unwrapping, or a |
On a meta note, I feel bad being the chief nay-sayer here. I have been and am working with @mpickering on other things, for example. And I completely understand it's been frustrating for him to see this to drag on and on, especially when it seemed on the verge of acceptance a few times over already, only to go back into limbo for another month+. I take no joy in dragging it on further. But just because a proposal is overdue getting resolved doesn't mean it deserves to be accepted. I am really surprised there really hasn't been much discussion of the motivation, one way or the other, from others. Can somebody who is not a current TTH user say why this is compelling to them? |
You can't get into the situation of a segfault by using
I am talking about when I implemented Overloaded Quotations internally to the compiler, not the Haskell implementation. In that case you have convenient access to a wrapper which applies the In this thread, a large proportion of known users indicate that this change would be beneficial and yet I have spent an inordinate amount of emotional energy attempting to make it happen. Please can we move on from this proposal! |
Personally, I don't find this proposal compelling at all. Let's consider the three points of the motivation section:
So, no part of the motivation seems compelling to me. That said, I don't feel too strongly either way, whereas @mpickering seems to care about this change, so I try not to object too strenuously or prevent this proposal from getting accepted. If there was voting I'd say it's a weak -0.5 from me for technical reasons, but +1 for social reasons (let people have their changes), totaling +0.5 |
-Code [|| 1 ||] :* Code [|| True ||] :* Code [|| "foo" ||] :* Nil
+[|| 1 ||] :* [|| True ||] :* [|| "foo" |]] :* Nil having written (yet unpublished) library which has code like that, the upper variant is simply not user friendly for public API (It would fine internally, and that's what it looks like now there). Similarly, the using of EDSL would be not nice with wrapping and unwrapping in the user code. Instances can be ugly, but writing
Defining wrappers: I see that one as an example for this proposal. This proposal removes the need to define these wrappers. EDIT: The changes in this proposal improve things for all TTH usages I can imagine myself, and don't make anything worse. I'm surprised by amount of low-quality counterexamples. |
@phadej The It's a reason to define different APIs for heterogeneous collections, not for TTH. As with
and then write it as:
The
That's a bold claim. It does make the use of monadic effects more complicated, hence the talk about Overall, I see how |
OK thank you so much @int-index --- I no longer need to feel like the sole crazy person out here. In fact agree with your -0.5 technical +1 social---this thing is not a big deal when way or the other, and I don't want to just waste @mpickering's time with tons of bureaucracy when TTH is basically being completely revamped. But, if we're going to merge this for the social reasons and not the technical ones, we should at least be explicit about that. Just taking the motivation on blind faith seems no good. |
I do think there is a desire to make the common cases for TTH as convenient as possible. But as @int-index says.
I agree completely. And let's keep in mind the risk here. If one feature takes some measures to be more usable, it's mostly fine. But If every feature (e.g. TTH) tries to hack around deficiencies in other features (e.g. heterogeneous collection ergonomics), we end up with a more complex language where disparate features view each other with suspicion rather than cooperate. For the sake of a whole, I am general pro not-hacking around things but waiting for fix the root causes of problems. We have to balance the completing interests of more beginner adoption of TTH now, vs a sensible API in conformance with the general idioms for writing Haskell later. Or, as we usually say, I am for avoiding success-at-all-costs. |
Thank you everyone for you input---I think we've clarified the proposal, and maybe even come up with some new ideas for future improvements/interesting research directions. Since this is a simple change, which names a common abstraction, and is clearly making current typed TH users happy, I think it is reasonable that we should accept it, and as the shepherd for this proposal I'll recommend to the committee that we do so. |
Code is now a newtype, so that it can be used to instantiate FreeExt. The tests still use Code for names throughout, which involves some extra boilerplate. But I think things'll become simpler again when we can use some proposals that have recently been adopted in GHC: Make Q (TExp a) into a newtype ghc-proposals/ghc-proposals#195 Overloaded Quotation Brackets ghc-proposals/ghc-proposals#246
Code is now a newtype, so that it can be used to instantiate FreeExt. The tests still use Code for names throughout, which involves some extra boilerplate. But I think things'll become simpler again when we can use some proposals that have recently been adopted in GHC: Make Q (TExp a) into a newtype ghc-proposals/ghc-proposals#195 Overloaded Quotation Brackets ghc-proposals/ghc-proposals#246
The proposal has been accepted; the following discussion is mostly of historic interest.
View rendered