Skip to content
  • Hjem
  • Seneste
  • Etiketter
  • Populære
  • Verden
  • Bruger
  • Grupper
Temaer
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Kollaps
FARVEL BIG TECH
  1. Forside
  2. Ikke-kategoriseret
  3. This story should terrify any open source project or company that allows LLM-generated code into their project.

This story should terrify any open source project or company that allows LLM-generated code into their project.

Planlagt Fastgjort Låst Flyttet Ikke-kategoriseret
25 Indlæg 20 Posters 0 Visninger
  • Ældste til nyeste
  • Nyeste til ældste
  • Most Votes
Svar
  • Svar som emne
Login for at svare
Denne tråd er blevet slettet. Kun brugere med emne behandlings privilegier kan se den.
  • david_chisnall@infosec.exchangeD This user is from outside of this forum
    david_chisnall@infosec.exchangeD This user is from outside of this forum
    david_chisnall@infosec.exchange
    wrote sidst redigeret af
    #1

    This story should terrify any open source project or company that allows LLM-generated code into their project.

    Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

    When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

    Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

    Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

    rzeta0@mathstodon.xyzR maddiem4@raphus.socialM cancername@mas.toC fluffykittycat@furry.engineerF joostvanderborg@mastodon.socialJ 13 Replies Last reply
    1
    0
    • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

      This story should terrify any open source project or company that allows LLM-generated code into their project.

      Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

      When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

      Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

      Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

      rzeta0@mathstodon.xyzR This user is from outside of this forum
      rzeta0@mathstodon.xyzR This user is from outside of this forum
      rzeta0@mathstodon.xyz
      wrote sidst redigeret af
      #2

      @david_chisnall

      "And that’s something that requires human judgement."

      someone in microsoft is coming up with a paid extra "ai compliance centre" which will aim to do this for you.

      inthehands@hachyderm.ioI 1 Reply Last reply
      0
      • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

        This story should terrify any open source project or company that allows LLM-generated code into their project.

        Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

        When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

        Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

        Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

        maddiem4@raphus.socialM This user is from outside of this forum
        maddiem4@raphus.socialM This user is from outside of this forum
        maddiem4@raphus.social
        wrote sidst redigeret af
        #3

        @david_chisnall

        bitanath@mastodon.socialB 1 Reply Last reply
        0
        • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

          This story should terrify any open source project or company that allows LLM-generated code into their project.

          Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

          When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

          Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

          Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

          cancername@mas.toC This user is from outside of this forum
          cancername@mas.toC This user is from outside of this forum
          cancername@mas.to
          wrote sidst redigeret af
          #4

          @david_chisnall s/Gemini/Claude/?

          1 Reply Last reply
          0
          • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

            This story should terrify any open source project or company that allows LLM-generated code into their project.

            Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

            When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

            Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

            Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

            fluffykittycat@furry.engineerF This user is from outside of this forum
            fluffykittycat@furry.engineerF This user is from outside of this forum
            fluffykittycat@furry.engineer
            wrote sidst redigeret af
            #5

            @david_chisnall critical support for AI companies totally destroying copyright law as one of it's casualties. Hopefully we can count that as a positive outcome of all this while eating our wacky cake and water pies after the crash

            dalias@hachyderm.ioD 1 Reply Last reply
            0
            • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

              This story should terrify any open source project or company that allows LLM-generated code into their project.

              Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

              When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

              Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

              Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

              joostvanderborg@mastodon.socialJ This user is from outside of this forum
              joostvanderborg@mastodon.socialJ This user is from outside of this forum
              joostvanderborg@mastodon.social
              wrote sidst redigeret af
              #6

              @david_chisnall It’s absolutely an issue, in this case I’d be wary of the source, the author doesn’t have the best track record as far as being thruthful about this app goes: https://daringfireball.net/2026/08/retraction_app_store_rejection_of_the_week#:~:text=frustrated%20by%20the,the%20domain%20darkhours.app.

              stuartmarks@mastodon.socialS 1 Reply Last reply
              0
              • maddiem4@raphus.socialM maddiem4@raphus.social

                @david_chisnall

                bitanath@mastodon.socialB This user is from outside of this forum
                bitanath@mastodon.socialB This user is from outside of this forum
                bitanath@mastodon.social
                wrote sidst redigeret af
                #7

                @MaddieM4 @david_chisnall isnt this the app that got rejected from the App Store for being an astrology app passing off as an astronomy app?

                1 Reply Last reply
                0
                • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

                  This story should terrify any open source project or company that allows LLM-generated code into their project.

                  Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

                  When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

                  Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

                  Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

                  giacomo@snac.tesio.itG This user is from outside of this forum
                  giacomo@snac.tesio.itG This user is from outside of this forum
                  giacomo@snac.tesio.it
                  wrote sidst redigeret af
                  #8
                  @david_chisnall@infosec.exchange

                  Well, ultimately, it's how the #PayToTrain #BusinessModel works.

                  The original "author" #vibecoded the project using the very same #agent (#claude).
                  1 Reply Last reply
                  0
                  • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

                    This story should terrify any open source project or company that allows LLM-generated code into their project.

                    Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

                    When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

                    Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

                    Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

                    lexfeathers@tenforward.socialL This user is from outside of this forum
                    lexfeathers@tenforward.socialL This user is from outside of this forum
                    lexfeathers@tenforward.social
                    wrote sidst redigeret af
                    #9

                    @david_chisnall jesus fucking christ

                    1 Reply Last reply
                    0
                    • fluffykittycat@furry.engineerF fluffykittycat@furry.engineer

                      @david_chisnall critical support for AI companies totally destroying copyright law as one of it's casualties. Hopefully we can count that as a positive outcome of all this while eating our wacky cake and water pies after the crash

                      dalias@hachyderm.ioD This user is from outside of this forum
                      dalias@hachyderm.ioD This user is from outside of this forum
                      dalias@hachyderm.io
                      wrote sidst redigeret af
                      #10

                      @fluffykittycat @david_chisnall The idea that you're going to get a destruction of copyright that applies equally to *you* as to VC-backed monsters is a delusional fantasy.

                      Even if you did get this in some jurisdictions, you would not get it uniformly across all jurisdictions.

                      Free software needs to be free regardless of whose version or extent of copyright abolution a user might be subject to the jurisdiction of.

                      fluffykittycat@furry.engineerF 1 Reply Last reply
                      0
                      • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

                        This story should terrify any open source project or company that allows LLM-generated code into their project.

                        Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

                        When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

                        Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

                        Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

                        nev@flipping.rocksN This user is from outside of this forum
                        nev@flipping.rocksN This user is from outside of this forum
                        nev@flipping.rocks
                        wrote sidst redigeret af
                        #11

                        @david_chisnall I should note that the inadvertently plagiarized web app is *also* coded with Claude. Who knows what parts of previous projects it incorporates?

                        1 Reply Last reply
                        0
                        • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

                          This story should terrify any open source project or company that allows LLM-generated code into their project.

                          Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

                          When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

                          Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

                          Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

                          mancube@pnw.zoneM This user is from outside of this forum
                          mancube@pnw.zoneM This user is from outside of this forum
                          mancube@pnw.zone
                          wrote sidst redigeret af
                          #12

                          @david_chisnall dang double bummer that that person is a sloperator. some of his stuff seemed fun in passing and i never really checked into it

                          1 Reply Last reply
                          0
                          • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

                            This story should terrify any open source project or company that allows LLM-generated code into their project.

                            Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

                            When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

                            Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

                            Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

                            tomjennings@tldr.nettime.orgT This user is from outside of this forum
                            tomjennings@tldr.nettime.orgT This user is from outside of this forum
                            tomjennings@tldr.nettime.org
                            wrote sidst redigeret af
                            #13

                            @david_chisnall

                            This is a critical "detail", seemingly obvious once you've made it clear like this, that i suspect small hobbiest type users are unaware of.

                            I assume large corporate users know this already and are ok eith just stealing copyrighted works and hiding behind lawyers and legal costs.

                            Thsnk you for this.

                            1 Reply Last reply
                            0
                            • dalias@hachyderm.ioD dalias@hachyderm.io

                              @fluffykittycat @david_chisnall The idea that you're going to get a destruction of copyright that applies equally to *you* as to VC-backed monsters is a delusional fantasy.

                              Even if you did get this in some jurisdictions, you would not get it uniformly across all jurisdictions.

                              Free software needs to be free regardless of whose version or extent of copyright abolution a user might be subject to the jurisdiction of.

                              fluffykittycat@furry.engineerF This user is from outside of this forum
                              fluffykittycat@furry.engineerF This user is from outside of this forum
                              fluffykittycat@furry.engineer
                              wrote sidst redigeret af
                              #14

                              @dalias @david_chisnall you're right, but the damage to the legitimacy of copyright ideology is still happening.

                              bms@mastodon.bsd.cafeB 1 Reply Last reply
                              0
                              • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

                                This story should terrify any open source project or company that allows LLM-generated code into their project.

                                Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

                                When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

                                Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

                                Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

                                dpnash@c.imD This user is from outside of this forum
                                dpnash@c.imD This user is from outside of this forum
                                dpnash@c.im
                                wrote sidst redigeret af
                                #15

                                @david_chisnall The "Mea Culpa" author's *mea culpa* doesn't pass the smell test.

                                The first sign of trouble is the identical name for a nearly identical app.

                                When I wrote a small astronomy calculation library (under a name that would be familiar to astronomers, so likely not unique) I made *damn sure* that there was nothing else out there on Github or Codeberg with the same name that was sufficiently like it to incur possible trademark issues.

                                dpnash@c.imD 1 Reply Last reply
                                0
                                • dpnash@c.imD dpnash@c.im

                                  @david_chisnall The "Mea Culpa" author's *mea culpa* doesn't pass the smell test.

                                  The first sign of trouble is the identical name for a nearly identical app.

                                  When I wrote a small astronomy calculation library (under a name that would be familiar to astronomers, so likely not unique) I made *damn sure* that there was nothing else out there on Github or Codeberg with the same name that was sufficiently like it to incur possible trademark issues.

                                  dpnash@c.imD This user is from outside of this forum
                                  dpnash@c.imD This user is from outside of this forum
                                  dpnash@c.im
                                  wrote sidst redigeret af
                                  #16

                                  @david_chisnall Also, as a longstanding amateur astronomer, I don't particularly trust the original app (https://darkhours.app) that he almost certainly plagiarized. I plugged in the coordinates for a location I did a little light observing from last month, which it claimed was "Bortle 3" (very dark sky, minimally affected by city lights). That's broadly consistent with what I saw myself -- some light pollution domes from cities in the distance, but very dark overhead. Moving just a little over *one hundred meters* to the north changed it to "Bortle 6" (suburban levels of light pollution; very much visibly lighter than Bortle 3). That's ... not how light pollution works, *at all*.

                                  siwek@social.tooinconsistent.comS 1 Reply Last reply
                                  0
                                  • joostvanderborg@mastodon.socialJ joostvanderborg@mastodon.social

                                    @david_chisnall It’s absolutely an issue, in this case I’d be wary of the source, the author doesn’t have the best track record as far as being thruthful about this app goes: https://daringfireball.net/2026/08/retraction_app_store_rejection_of_the_week#:~:text=frustrated%20by%20the,the%20domain%20darkhours.app.

                                    stuartmarks@mastodon.socialS This user is from outside of this forum
                                    stuartmarks@mastodon.socialS This user is from outside of this forum
                                    stuartmarks@mastodon.social
                                    wrote sidst redigeret af
                                    #17

                                    @joostvanderborg Came here to post the DF article but I see you posted it already. I agree that Godier’s credibility is questionable.

                                    Maybe that changes some of the facts behind @david_chisnall’s post but the issues it raises around provenance and copyright of LLM remain, regardless of Godier’s intent in this particular incident. The question I have is whether Godier actually used Claude or whether he himself made the copy and is trying to blame Claude.

                                    1 Reply Last reply
                                    0
                                    • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

                                      This story should terrify any open source project or company that allows LLM-generated code into their project.

                                      Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

                                      When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

                                      Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

                                      Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

                                      cap_ybarra@beige.partyC This user is from outside of this forum
                                      cap_ybarra@beige.partyC This user is from outside of this forum
                                      cap_ybarra@beige.party
                                      wrote sidst redigeret af
                                      #18

                                      @david_chisnall @zzt honestly this story will not terrify anywhere near as many people as it should

                                      1 Reply Last reply
                                      0
                                      • david_chisnall@infosec.exchangeD david_chisnall@infosec.exchange

                                        This story should terrify any open source project or company that allows LLM-generated code into their project.

                                        Someone created a new app, using Claude. Only it wasn’t a new app, it was clearly plagiarised because it happened that there was an example in the training set that exactly matched the requirements. The similarity was well within the range that courts have previously used to determine a derived work.

                                        When you have this level of similarity, the requirement comes to you to prove that there was no way that the original work could have flowed to your project. Companies that have this concern usually do it by ensuring that no one on the team has been exposed to the original and that the code for the original never goes near their systems. But when one of the systems that you use is a language model trained on, among other things, all of the open-source code that it could scrape (and which does not disclose its training set), being able to prove that there was no path from some other codebase to yours is impossible.

                                        Just because the US copyright office has ruled that you, as the person promoting an LLM, cannot assert copyright on the output, does not mean that someone else can’t. If you take a DVD and transcode it to H.264, there is no creative step and so the new copy is not something subject to independent copyright, but it is a derived work of the DVD (itself a lower-quality derived work of the original masters) and so subject to the same copyright.

                                        Importantly in this story, the person prompting Claude had no idea that the original app existed. To safely use the code, they would need to search everything in the training data and discard outputs that would meet the bar of being substantially similar. And that’s something that requires human judgement.

                                        eff3rvecenteden@mastodon.socialE This user is from outside of this forum
                                        eff3rvecenteden@mastodon.socialE This user is from outside of this forum
                                        eff3rvecenteden@mastodon.social
                                        wrote sidst redigeret af
                                        #19

                                        @david_chisnall local LLM's are the new printing press for a house. Major companies only have a monopoly bc there's an illusion they are the only ones capable

                                        Get your opensource model today guys!

                                        1 Reply Last reply
                                        0
                                        • dpnash@c.imD dpnash@c.im

                                          @david_chisnall Also, as a longstanding amateur astronomer, I don't particularly trust the original app (https://darkhours.app) that he almost certainly plagiarized. I plugged in the coordinates for a location I did a little light observing from last month, which it claimed was "Bortle 3" (very dark sky, minimally affected by city lights). That's broadly consistent with what I saw myself -- some light pollution domes from cities in the distance, but very dark overhead. Moving just a little over *one hundred meters* to the north changed it to "Bortle 6" (suburban levels of light pollution; very much visibly lighter than Bortle 3). That's ... not how light pollution works, *at all*.

                                          siwek@social.tooinconsistent.comS This user is from outside of this forum
                                          siwek@social.tooinconsistent.comS This user is from outside of this forum
                                          siwek@social.tooinconsistent.com
                                          wrote sidst redigeret af
                                          #20

                                          @dpnash @david_chisnall from quick peek at github this app is also fully vibe coded slop, so I wouldn't be surprised that it's full of shit.

                                          Also there's a non-zero chance that the app that's supposedly plagiarised is a plagiarism in of itself, and they simply both put in the same prompts and got same output..

                                          It's slop all the way down.

                                          dpnash@c.imD 1 Reply Last reply
                                          0
                                          Svar
                                          • Svar som emne
                                          Login for at svare
                                          • Ældste til nyeste
                                          • Nyeste til ældste
                                          • Most Votes


                                          • Log ind

                                          • Har du ikke en konto? Tilmeld

                                          • Login or register to search.
                                          Powered by NodeBB Contributors
                                          Graciously hosted by data.coop
                                          • First post
                                            Last post
                                          0
                                          • Hjem
                                          • Seneste
                                          • Etiketter
                                          • Populære
                                          • Verden
                                          • Bruger
                                          • Grupper