From boomer3200 at gmail.com Sun Dec 1 13:28:42 2024 From: boomer3200 at gmail.com (Dan Plassche) Date: Sat, 30 Nov 2024 22:28:42 -0500 (EST) Subject: [TUHS] Typesetter C Compiler and Troff (Re: Re: v6 Unix Documents) In-Reply-To: References: <20241018135806.ysgog7qde6xjodtu@illithid> <179b0bb4-73d1-fc81-b2d3-68b651e58bbd@gmail.com> Message-ID: > Date: Sun, 20 Oct 2024 22:47:43 > From: Jonathan Gray > > > I adapted Tim Newsham's v6 install scripts for PWB if you'd > like to run it on simh. > > https://github.com/jonathangray/pwb/ Thanks, very helpful and worked well. Since I was able to follow the setup script and rebuild the c versions of nroff/troff on PWB 1.0, I took a look at the new c compiler for differences with the UNSW copy. It turns out that the UNSW copy of the c compiler was definitely a earlier version than in PWB. The most noticeable difference in the UNSW stdio library was calloc.c not using malloc. The most major omission in the c compiler was casts and there were also some local changes to cc.c. Based on those differences, I'm thinking that the UNSW copy appears to be from late 1976 to early 1977. I was able to rebuild both the UNSW and the native PWB compiler on PWB 1.0, but not to backport either to vanilla v6. Best, Dan From jnc at mercury.lcs.mit.edu Sun Dec 1 14:21:31 2024 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Sat, 30 Nov 2024 23:21:31 -0500 (EST) Subject: [TUHS] Typesetter C Compiler and Troff (Re: Re: v6 Unix Documents) Message-ID: <20241201042131.E7DD318C075@mercury.lcs.mit.edu> > From: > I was able to rebuild both the UNSW and the native PWB compiler on PWB > 1.0, but not to backport either to vanilla v6. Any idea what the problem was? I'm curious, because we ran a version of the Typesetter compiler on the MIT systems, which ran an enhanced V6. Noel From ron at ronnatalie.com Mon Dec 2 01:40:58 2024 From: ron at ronnatalie.com (Ron Natalie) Date: Sun, 01 Dec 2024 15:40:58 +0000 Subject: [TUHS] Typesetter C Compiler and Troff (Re: Re: v6 Unix Documents) In-Reply-To: <20241201042131.E7DD318C075@mercury.lcs.mit.edu> References: <20241201042131.E7DD318C075@mercury.lcs.mit.edu> Message-ID: Agreed, the typesetter C ran on a virgin V6 system. That was the whole point. It came out before most of us got real V7s. ------ Original Message ------ >From "Noel Chiappa" To tuhs at tuhs.org Cc jnc at mercury.lcs.mit.edu Date 11/30/2024 11:21:31 PM Subject [TUHS] Re: Typesetter C Compiler and Troff (Re: Re: v6 Unix Documents) > > From: > > > I was able to rebuild both the UNSW and the native PWB compiler on PWB > > 1.0, but not to backport either to vanilla v6. > >Any idea what the problem was? I'm curious, because we ran a version of the >Typesetter compiler on the MIT systems, which ran an enhanced V6. > > Noel > From clemc at ccc.com Mon Dec 2 03:24:16 2024 From: clemc at ccc.com (Clem Cole) Date: Sun, 1 Dec 2024 12:24:16 -0500 Subject: [TUHS] Typesetter C Compiler and Troff (Re: Re: v6 Unix Documents) In-Reply-To: References: <20241201042131.E7DD318C075@mercury.lcs.mit.edu> Message-ID: Exactly -- remember Dennis and Brian wrote K&R in 1978 on V6, not PWB and not V7 (or anything from Summit like TS). When Brian's typesetter independent troff was released, that updated language was needed. That was what we referred to as "typesetter C". Brian and Dennis would have put together that kit. My >>guess<< is that Mashey et al. and the Summit folks forked Dennis compiler somewhere before this time -- so what got packaged with the typesetter kit vs what got packed with Mashey and the team's kit are close but >>slightly<< different. ᐧ On Sun, Dec 1, 2024 at 10:49 AM Ron Natalie wrote: > Agreed, the typesetter C ran on a virgin V6 system. That was the whole > point. It came out before most of us got real V7s. > > > > ------ Original Message ------ > From "Noel Chiappa" > To tuhs at tuhs.org > Cc jnc at mercury.lcs.mit.edu > Date 11/30/2024 11:21:31 PM > Subject [TUHS] Re: Typesetter C Compiler and Troff (Re: Re: v6 Unix > Documents) > > > > From: > > > > > I was able to rebuild both the UNSW and the native PWB compiler on > PWB > > > 1.0, but not to backport either to vanilla v6. > > > >Any idea what the problem was? I'm curious, because we ran a version of > the > >Typesetter compiler on the MIT systems, which ran an enhanced V6. > > > > Noel > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Mon Dec 2 03:27:04 2024 From: clemc at ccc.com (Clem Cole) Date: Sun, 1 Dec 2024 12:27:04 -0500 Subject: [TUHS] Typesetter C Compiler and Troff (Re: Re: v6 Unix Documents) In-Reply-To: References: <20241201042131.E7DD318C075@mercury.lcs.mit.edu> Message-ID: I should point out to add to the confusion, originally Mashey's team - the PWB 1.0 folks are != USG (Summit folks) although the later took over the PWB work as I understand it. ᐧ On Sun, Dec 1, 2024 at 12:24 PM Clem Cole wrote: > Exactly -- remember Dennis and Brian wrote K&R in 1978 on V6, not PWB and > not V7 (or anything from Summit like TS). When Brian's typesetter > independent troff was released, that updated language was needed. That > was what we referred to as "typesetter C". Brian and Dennis would have put > together that kit. My >>guess<< is that Mashey et al. and the Summit > folks forked Dennis compiler somewhere before this time -- so what got > packaged with the typesetter kit vs what got packed with Mashey and the > team's kit are close but >>slightly<< different. > ᐧ > > On Sun, Dec 1, 2024 at 10:49 AM Ron Natalie wrote: > >> Agreed, the typesetter C ran on a virgin V6 system. That was the whole >> point. It came out before most of us got real V7s. >> >> >> >> ------ Original Message ------ >> From "Noel Chiappa" >> To tuhs at tuhs.org >> Cc jnc at mercury.lcs.mit.edu >> Date 11/30/2024 11:21:31 PM >> Subject [TUHS] Re: Typesetter C Compiler and Troff (Re: Re: v6 Unix >> Documents) >> >> > > From: >> > >> > > I was able to rebuild both the UNSW and the native PWB compiler >> on PWB >> > > 1.0, but not to backport either to vanilla v6. >> > >> >Any idea what the problem was? I'm curious, because we ran a version of >> the >> >Typesetter compiler on the MIT systems, which ran an enhanced V6. >> > >> > Noel >> > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sjenkin at canb.auug.org.au Wed Dec 4 13:17:49 2024 From: sjenkin at canb.auug.org.au (sjenkin at canb.auug.org.au) Date: Wed, 4 Dec 2024 14:17:49 +1100 Subject: [TUHS] After 50 years, what has the Impact of Unix been? Message-ID: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> I was looking at the question of “impact of Unix" and thought that: initiating (Portable) Open Source Software including the BSD TCP/IP & Berkeley sockets libraries, the Unix -> Minix -> Linux -> Android sequence and BSD -> NeXtstep -> OS/X -> iOS sequence, plus running the Top 500 supercomputers and most of the Top 500 websites, including the infrastructure for trillion dollars companies, Facebook, Amazon (Netflix uses them), and Google plus so many embedded Linux / NetBSD based appliances even going into space - on small experiments or driving SpaceX’s Falcon 9 & presumably Starship, would be a slam-dunk for “really high impact” - dominating everywhere important, except Windows desktops. Unix wasn’t just a ’research project’, it was the result of years of work by a team of very capable, professional programmers, who weren’t looking for kudos or recognition, nor trying to add every conceivable ‘feature’ possible, but the inverse: how _small_ could it be and still be enough. When it left the Labs, Unix wasn’t just “Performant”, it came with a very useful set of tools for creating other tools (‘developing’) and while the kernel wasn’t perfect (some ‘panic’s), it was of “Production Quality”. For v6, I believe there were patches for 50-100 bugs circulating, perhaps the first demonstration of “no bug is intractable with ‘many eyeballs’”. All that in under 5 years of ‘development’, with the “initial release” stable & performant enough for cash-strapped Universities to gamble their reputations & budgets on running it. Imagine the grief academics would’ve got if their basic teaching systems failed continuously! This adoption path pushed Unix through another barrier: ’Security’ - with a lot of bright, bored & curious students banging on it as hard as they could for bragging rights. How long, in releases or years, did it take for other O/S’s to hit the “very stable” benchmark? I don’t know enough of Linux to answer that definitively, the *BSD’s grew there through usage and contribution, while Microsoft NT derivates widely suffered “Blue Screen of Death” for years. Even now, MS-Windows has serious Security / compromise issues, like the highly visible, global “Crowdstrike” event. Not a break-in or takeover, but an own-goal from Security perimeter control. ========== I now think Unix has had a much larger, direct and important impact - the C language and associated tools & libraries that begat modern toolchains and endless portability across platforms. In 1991, Bill Plauger had a year sabbatical at UNSW in Sydney, and happened to say : “C is wallpaper - people expect it everywhere”. C gained formal recognition with the POSIX standard, satisfying conservative users / enterprises that it wasn’t the work of a bunch of Hippies or ill-disciplined Hackers. Even Microsoft wrote 32-bit Windows NT in C, I presume starting by writing it’s own compiler and toolchain to start. Borland, Watcom and many others - including Microsoft - offered (Visual) C compile & build environments for Windows, directly responsible for creating the ’shrink-wrap’ third party software market that drove sales of Windows and x86 machines. Nobody had seen a market for a billion systems before, nor sold 300M+ CPU’s in a single year. People don’t buy Silicon Chips or nice Boxes, they buy Applications that solve their problems: Software drives Sales of Hardware - something that IBM deeply understood first with first the 1401 line, then 360-series. The other ’small’ achievement of C and Unix was creating the market for RISC chips. MIPS in the mid-1980’s was only able to design and build the first commercial RISC chip because it knew it could port Unix to it and find an immediate market - not at zero-cost, but a tiny fraction of what every other Vendor had done before reinventing the wheel from scratch to provide incompatible O/S & tools for their hardware. Unix on MIPS not only came with a host of proven software, that a large pool of people knew how to use, but it arrived as “Production Quality” - the porting team had to test their parts - compiler, linker, libraries - hard, but could trust the existing high-quality codebase. In "A New Golden Age for Computer Architecture”, 2019 by Hennessy & Patterson, make an aside: In today's post-PC era, x86 shipments have fallen almost 10% per year since the peak in 2011, while chips with RISC processors have skyrocketed to 20 billion. Today, 99% of 32-bit and 64-bit processors are RISC. i suggest this goes back to PCC followed by the MIPS R2000 - made possible by Dennis’ C language. The 1977 invention of ‘pcc’ and rewriting of Unix for cross-machine portability was the first time I’m aware of this being done. ( Miller @ UoW did a one-off hack, not to devalue his work, he ported, didn’t invent a multi-target portable compiler ) One of the effects of “portable C” was creating whole new industries for third party software developers or enabling niche products, like CISCO routers and the many embedded devices. C and Unix came with the tools to create new languages and new tools. AWK, sed (!) and shells are obvious examples, with Perl, Python & PHP very big in Internet of 2000. C was a new class of language - a tool to create tools. It creates a perfect mechanism to bootstrap any new language, tool or product, allowing to be refined & used enough to become reliable before being made self-hosting. Very widely used languages such as Python are written in C. ORACLE achieved its market dominance by providing ‘portability’ - exactly the same on every platform. Underpinned by portable C. The original 1127 team went on to create other systems and languages, not the least being a new Software Engineering tool, “Go” / golang, addressing a whole slew of deficiencies in the C/C++ approach and We’d have no Internet today without Apache written in C and being ported to every environment. Also, there’s a connection between C and ‘modern’ Software Engineering - distributed Repositories, automated builds & regression tests, and the many toolchains and tools used. They tended to be built in C to address problems (Open Source) developers were finding with existing toolchains. ‘make’ arose at Bell Labs to automate builds, along with PWB and Writers Workbench. There’s two questions / observations about 50 years of C in broad use: - Just how much C is out there and used ‘in production’? - C is ‘obviously’ a product of the 1970’s, not reflecting needs of modern hardware, networks, storage and systems, but _what_ can replace it? There is simply too much critical code written in C to convert it to another ‘better, modern’ language. Any new language that is a simple 1:1 rewriting of C cannot address any of the deficiencies, while any incompatible language requires redesign and reimplementation of everything - an unachievable goal. The Linux Kernel’s “rust” project shows the extent of the problem - even with the best team of the best developers, its a mammoth undertaking, with uncertain payoffs, unquantifiable effort & deadlines. My thesis is that portable, standard C: - not only co-evolved with other tools & needs to create the Modern Software Engineering environment, the basis for multiple Trillion dollar enterprises (FAANG) but - drove the biggest, most profitable software market ever seen (Wintel) - which drove sales volume of x86 chips (& DRAM, motherboards, LAN, GPU, monitors, peripherals…) over 2-3 decades, - which drove Silicon Valley, paying for new generations of Fabs and lowering chip prices further & further - and eventually created the Fabless RISC CPU company, which in the Post-PC era absolutely dominates chip sales. No Software, no Silicon… Gordon Moore, in an early comment on his 1968 startup with Robert Noyce, said: “we are the real revolutionaries" (vs Hippies & 1967 Summer of Love). I think Ken & Dennis [ and 1127/ Bell Labs folk ] can say the same. ========== I’ve written some notes, with links to Programming Languages, especially Jean Sammet’s Histories, and would like some critiques, suggestions & corrections if people have time and interest. Unix and C are intimately related - neither was possible or useful without the other. i think there’s an interesting article in there, but I’m not sure I have what it takes to write it, not in a finite time :) Very happy to help anyone who does! Did-C-lang-create-modern-software-industry.txt steve jenkin 04 - dec - 2024 ========== -- Steve Jenkin, IT Systems and Design 0412 786 915 (+61 412 786 915) PO Box 38, Kippax ACT 2615, AUSTRALIA mailto:sjenkin at canb.auug.org.au http://members.tip.net.au/~sjenkin From tuhs at tuhs.org Wed Dec 4 13:57:46 2024 From: tuhs at tuhs.org (Jan Schaumann via TUHS) Date: Tue, 3 Dec 2024 22:57:46 -0500 Subject: [TUHS] The History of the BSD Daemon Message-ID: Marshall Kirk McKusick gave a talk on the history of the BSD Daemon: https://www.youtube.com/watch?v=AeDaD-CEzzg "This video tells the history of the BSD Daemon. It starts with the first renditions in the 1970s of the daemons that help UNIX systems provide services to users. These early daemons were the inspiration for the well-known daemon created by John Lasseter in the early 1980s that became synonymous with BSD as they adorned the covers of the first three editions of `The Design and Implementation of the BSD Operating System' textbooks. The talk will also highlight many of the shirt designs that featured the BSD Daemon." From imp at bsdimp.com Wed Dec 4 14:07:21 2024 From: imp at bsdimp.com (Warner Losh) Date: Tue, 3 Dec 2024 21:07:21 -0700 Subject: [TUHS] The History of the BSD Daemon In-Reply-To: References: Message-ID: On Tue, Dec 3, 2024, 8:57 PM Jan Schaumann via TUHS wrote: > Marshall Kirk McKusick gave a talk on the history of > the BSD Daemon: > > https://www.youtube.com/watch?v=AeDaD-CEzzg > > "This video tells the history of the BSD Daemon. It > starts with the first renditions in the 1970s of the > daemons that help UNIX systems provide services to > users. These early daemons were the inspiration for > the well-known daemon created by John Lasseter in the > early 1980s that became synonymous with BSD as they > adorned the covers of the first three editions of `The > Design and Implementation of the BSD Operating System' > textbooks. The talk will also highlight many of the > shirt designs that featured the BSD Daemon." > Great talk and awesome T Shirts... Warner > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Wed Dec 4 18:50:38 2024 From: imp at bsdimp.com (Warner Losh) Date: Wed, 4 Dec 2024 01:50:38 -0700 Subject: [TUHS] The History of the BSD Daemon In-Reply-To: References: Message-ID: https://youtu.be/7d7fwAE-2Aw?si=au0vDr2XAHfAJOc1 Has remastered sound... sounds a lot better to my ears. Warner On Tue, Dec 3, 2024, 8:57 PM Jan Schaumann via TUHS wrote: > Marshall Kirk McKusick gave a talk on the history of > the BSD Daemon: > > https://www.youtube.com/watch?v=AeDaD-CEzzg > > "This video tells the history of the BSD Daemon. It > starts with the first renditions in the 1970s of the > daemons that help UNIX systems provide services to > users. These early daemons were the inspiration for > the well-known daemon created by John Lasseter in the > early 1980s that became synonymous with BSD as they > adorned the covers of the first three editions of `The > Design and Implementation of the BSD Operating System' > textbooks. The talk will also highlight many of the > shirt designs that featured the BSD Daemon." > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc.donner at gmail.com Wed Dec 4 23:05:45 2024 From: marc.donner at gmail.com (Marc Donner) Date: Wed, 4 Dec 2024 08:05:45 -0500 Subject: [TUHS] After 50 years, what has the Impact of Unix been? In-Reply-To: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> Message-ID: I would add one more important impact: making data a first class component of the computing environment. With the notion of pipes it became possible to operate on data quickly and flexibly. There was nothing new from a fundamental capability point of view, but the ease with which one could construct pipelines enabled rapid experimentation and encouraged the development of pipe-able components to add to the tool set. I may not have articulated this as well as it deserves. Marc ===== nygeek.net mindthegapdialogs.com/home On Wed, Dec 4, 2024 at 3:43 AM wrote: > I was looking at the question of “impact of Unix" and thought that: > > initiating (Portable) Open Source Software including the BSD > TCP/IP & Berkeley sockets libraries, > the Unix -> Minix -> Linux -> Android sequence > and BSD -> NeXtstep -> OS/X -> iOS sequence, > plus running the Top 500 supercomputers and most of the Top 500 > websites, > including the infrastructure for trillion dollars companies, > Facebook, Amazon (Netflix uses them), and Google > plus so many embedded Linux / NetBSD based appliances > even going into space - on small experiments or driving SpaceX’s > Falcon 9 & presumably Starship, > > would be a slam-dunk for “really high impact” > - dominating everywhere important, except Windows desktops. > > Unix wasn’t just a ’research project’, it was the result of years of work > by a team of very capable, professional programmers, > who weren’t looking for kudos or recognition, nor trying to add every > conceivable ‘feature’ possible, but the inverse: > > how _small_ could it be and still be enough. > > When it left the Labs, Unix wasn’t just “Performant”, it came with a very > useful set of tools for creating other tools (‘developing’) > and while the kernel wasn’t perfect (some ‘panic’s), it was of “Production > Quality”. > > For v6, I believe there were patches for 50-100 bugs circulating, perhaps > the first demonstration of “no bug is intractable with ‘many eyeballs’”. > > All that in under 5 years of ‘development’, with the “initial release” > stable & performant enough for cash-strapped Universities > to gamble their reputations & budgets on running it. > Imagine the grief academics would’ve got if their basic teaching systems > failed continuously! > > This adoption path pushed Unix through another barrier: > > ’Security’ - with a lot of bright, bored & curious students > banging on it as hard as they could for bragging rights. > > How long, in releases or years, did it take for other O/S’s to hit the > “very stable” benchmark? > > I don’t know enough of Linux to answer that definitively, the *BSD’s grew > there through usage and contribution, > while Microsoft NT derivates widely suffered “Blue Screen of Death” for > years. > > Even now, MS-Windows has serious Security / compromise issues, like the > highly visible, global “Crowdstrike” event. > Not a break-in or takeover, but an own-goal from Security perimeter > control. > > ========== > > I now think Unix has had a much larger, direct and important impact > > - the C language and associated tools & libraries > that begat modern toolchains and endless portability > across platforms. > > In 1991, Bill Plauger had a year sabbatical at UNSW in Sydney, > and happened to say : > “C is wallpaper - people expect it everywhere”. > > C gained formal recognition with the POSIX standard, satisfying > conservative users / enterprises that it wasn’t the work of a bunch of > Hippies or ill-disciplined Hackers. > > Even Microsoft wrote 32-bit Windows NT in C, I presume starting by writing > it’s own compiler and toolchain to start. > > Borland, Watcom and many others - including Microsoft - offered (Visual) C > compile & build environments for Windows, > directly responsible for creating the ’shrink-wrap’ third party software > market that drove sales of Windows and x86 machines. > > Nobody had seen a market for a billion systems before, nor sold 300M+ > CPU’s in a single year. > > People don’t buy Silicon Chips or nice Boxes, they buy Applications that > solve their problems: > > Software drives Sales of Hardware > - something that IBM deeply understood first with first the 1401 > line, then 360-series. > > > The other ’small’ achievement of C and Unix was creating the market for > RISC chips. > MIPS in the mid-1980’s was only able to design and build the first > commercial RISC chip > because it knew it could port Unix to it and find an immediate market > - not at zero-cost, but a tiny fraction of what every other Vendor > had done before > reinventing the wheel from scratch to provide incompatible > O/S & tools for their hardware. > > Unix on MIPS not only came with a host of proven software, that a large > pool of people knew how to use, > but it arrived as “Production Quality” - the porting team had to test > their parts - compiler, linker, libraries - hard, but could trust the > existing high-quality codebase. > > In "A New Golden Age for Computer Architecture”, 2019 by Hennessy & > Patterson, > make an aside: > > In today's post-PC era, x86 shipments have fallen almost 10% per > year since the peak in 2011, > while chips with RISC processors have skyrocketed to 20 billion. > > Today, 99% of 32-bit and 64-bit processors are RISC. > > i suggest this goes back to PCC followed by the MIPS R2000 - made possible > by Dennis’ C language. > > The 1977 invention of ‘pcc’ and rewriting of Unix for cross-machine > portability was the first time I’m aware of this being done. > ( Miller @ UoW did a one-off hack, not to devalue his work, he ported, > didn’t invent a multi-target portable compiler ) > > One of the effects of “portable C” was creating whole new industries for > third party software developers > or enabling niche products, like CISCO routers and the many embedded > devices. > > C and Unix came with the tools to create new languages and new tools. > AWK, sed (!) and shells are obvious examples, with Perl, Python & PHP very > big in Internet of 2000. > > C was a new class of language - a tool to create tools. > It creates a perfect mechanism to bootstrap any new language, tool or > product, > allowing to be refined & used enough to become reliable before being made > self-hosting. > > Very widely used languages such as Python are written in C. > ORACLE achieved its market dominance by providing ‘portability’ - exactly > the same on every platform. > Underpinned by portable C. > > The original 1127 team went on to create other systems and languages, > not the least being a new Software Engineering tool, “Go” / golang, > addressing a whole slew of deficiencies in the C/C++ approach and > > We’d have no Internet today without Apache written in C and being ported > to every environment. > > Also, there’s a connection between C and ‘modern’ Software Engineering > - distributed Repositories, automated builds & regression tests, and the > many toolchains and tools used. > > They tended to be built in C to address problems (Open Source) developers > were finding with existing toolchains. > ‘make’ arose at Bell Labs to automate builds, along with PWB and Writers > Workbench. > > There’s two questions / observations about 50 years of C in broad use: > > - Just how much C is out there and used ‘in production’? > > - C is ‘obviously’ a product of the 1970’s, not reflecting needs > of modern hardware, networks, storage and systems, > but _what_ can replace it? > > There is simply too much critical code written in C to > convert it to another ‘better, modern’ language. > Any new language that is a simple 1:1 rewriting of C > cannot address any of the deficiencies, > while any incompatible language requires redesign and > reimplementation of everything - an unachievable goal. > > The Linux Kernel’s “rust” project shows the extent of the problem > - even with the best team of the best developers, its a mammoth > undertaking, with uncertain payoffs, unquantifiable effort & deadlines. > > My thesis is that portable, standard C: > > - not only co-evolved with other tools & needs to create the > Modern Software Engineering environment, the basis for multiple Trillion > dollar enterprises (FAANG) > but > - drove the biggest, most profitable software market ever seen > (Wintel) > - which drove sales volume of x86 chips (& DRAM, motherboards, > LAN, GPU, monitors, peripherals…) over 2-3 decades, > - which drove Silicon Valley, paying for new generations of Fabs > and lowering chip prices further & further > - and eventually created the Fabless RISC CPU company, > which in the Post-PC era absolutely dominates chip sales. > > No Software, no Silicon… > > Gordon Moore, in an early comment on his 1968 startup with Robert Noyce, > said: > > “we are the real revolutionaries" (vs Hippies & 1967 Summer of > Love). > > I think Ken & Dennis [ and 1127/ Bell Labs folk ] can say the same. > > ========== > > I’ve written some notes, with links to Programming Languages, especially > Jean Sammet’s Histories, > and would like some critiques, suggestions & corrections if people have > time and interest. > > Unix and C are intimately related - neither was possible or useful without > the other. > > i think there’s an interesting article in there, but I’m not sure I have > what it takes to write it, not in a finite time :) > Very happy to help anyone who does! > > Did-C-lang-create-modern-software-industry.txt > < > https://drive.google.com/file/d/1k936sgqHc-vHBvfCdLoSxFhdT9NaijU2/view?usp=sharing > > > > steve jenkin > 04 - dec - 2024 > > ========== > -- > Steve Jenkin, IT Systems and Design > 0412 786 915 (+61 412 786 915) > PO Box 38, Kippax ACT 2615, AUSTRALIA > > mailto:sjenkin at canb.auug.org.au http://members.tip.net.au/~sjenkin > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ches at cheswick.com Wed Dec 4 23:40:28 2024 From: ches at cheswick.com (William Cheswick) Date: Wed, 4 Dec 2024 08:40:28 -0500 Subject: [TUHS] After 50 years, what has the Impact of Unix been? In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> Message-ID: This alludes to an important step that should be mentioned. Plan 9 was a rethinking of the whole Unix world. Though it didn’t spread like C, many of its ideas were adopted elsewhere. And its approach to multi-machine compilation and tools is clearly reflected in the design and implementation of golang. ches > On Dec 4, 2024, at 8:05 AM, Marc Donner wrote: > > > The original 1127 team went on to create other systems and languages, > not the least being a new Software Engineering tool, “Go” / golang, > addressing a whole slew of deficiencies in the C/C++ approach and > From rich.salz at gmail.com Thu Dec 5 01:02:14 2024 From: rich.salz at gmail.com (Rich Salz) Date: Wed, 4 Dec 2024 10:02:14 -0500 Subject: [TUHS] After 50 years, what has the Impact of Unix been? In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> Message-ID: > > I would add one more important impact: making data a first class component > of the computing environment. > Related to that is getting rid of "file types". I remember at a "lessons of Unix" panel at a Usenix, that someone from the Labs gave an example of a situation where the operating system treated a file as more than just a set of bytes (think IBM, VMS, etc). Read a source file (required them to do a bunch of file-conversion operations), write an output, perhaps changing a variable name from 'foo' to 'bar', and then compiling the output, which also required a bunch of file-conversion operations. -------------- next part -------------- An HTML attachment was scrubbed... URL: From douglas.mcilroy at dartmouth.edu Thu Dec 5 05:07:07 2024 From: douglas.mcilroy at dartmouth.edu (Douglas McIlroy) Date: Wed, 4 Dec 2024 14:07:07 -0500 Subject: [TUHS] After 50 years, what has the Impact of Unix been? Message-ID: Rich's remark about file types reminded me of an ancient non-Unix story that I sent to Coff. Doug From johnl at taugh.com Thu Dec 5 13:08:43 2024 From: johnl at taugh.com (John Levine) Date: 4 Dec 2024 22:08:43 -0500 Subject: [TUHS] After 50 years, what has the Impact of Unix been? In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> Message-ID: <20241205030843.8552FAB1EDA5@ary.qy> It appears that Marc Donner said: >With the notion of pipes it became possible to operate on data quickly and >flexibly. There was nothing new from a fundamental capability point of >view, but the ease with which one could construct pipelines enabled rapid >experimentation and encouraged the development of pipe-able components to >add to the tool set. Pipes were invented at least three times I'm aware of, but what made them work so well in Unix is that they looked to the program the same as a file so any program could use them for input or output without special arrangements, and the shell made it easy to start two programs and pipe them together. The Dartmouth Time-Sharing System in the late 1960s had communication files which were essentially two-way pipes, but they were asymmetrical. One end, the slave end, looked like a file, but the other end, the master end, was different and the program had to know it was a com file. They were mostly used to pass terminal I/O between user programs at the slave end and SIMON at the master end, the terminal monitor that talked to the front end computer than ran the TTYs. They were invented again at IBM in the 1970s and described in this paper. I wrote them a letter, which they published, saying that Unix pipes did the same thing. https://dl.acm.org/doi/10.1147/sj.174.0383 R's, John From boomer3200 at gmail.com Thu Dec 5 16:38:34 2024 From: boomer3200 at gmail.com (Dan Plassche) Date: Thu, 5 Dec 2024 01:38:34 -0500 (EST) Subject: [TUHS] Typesetter C Compiler and Troff (Re: Re: v6 Unix Documents) In-Reply-To: <20241201042131.E7DD318C075@mercury.lcs.mit.edu> References: <20241201042131.E7DD318C075@mercury.lcs.mit.edu> Message-ID: On Sat, 30 Nov 2024, Noel Chiappa wrote: > > > I was able to rebuild both the UNSW and the native PWB compiler on PWB > > 1.0, but not to backport either to vanilla v6. > > Any idea what the problem was? I'm curious, because we ran a version of the > Typesetter compiler on the MIT systems, which ran an enhanced V6. > > Noel Yes, I have a general sense after making some progress building on v6 and further exploring the PWB 1.0 and UNSW sets as noted below. Would be interested to complete the process and add any details I am missing. I had to use the new c compiler and assembler from the Shoppa disk (nix_v6.rl02.gz) to bootstrap the build using the UNSW files on v6. The original typesetter c distribution was supposed to be for v6, which is my intended use case, but it appears the third-party copies we have available drifted: - the stock v6 compiler could not build the new "Portable I/O Library" (/lib/libS.a supporting stdio) from UNSW or PWB as the pre-requisite to building the new compiler. The code is almost all the same in both, although UNSW is missing tmpnam.c. - "old" cc gave a lot of basic syntax errors from the included stdio.h and the library files with some of the most common issues being redeclared items and classes. - I had to adjust the UNSW includes to point to the in-directory stdio.h file rather than the to be created include path (/usr/include/stdio.h from #include ) that was assumed to already exist (not on v6). - table.o for the stdio library appears to require the new as - PWB 1.0 builds already have ncc and are straightforward, but rely on makefiles and use syscalls that are not available on v6 - UNSW's set provides traditional run shell scripts, but incorporates features -- the Bourne shell colon no-op and as with an -o flag -- that are not on v6 - everything for stdio and the new compiler apart from c1 built cleanly using ncc with /xlib I am still tracking down why building c1 lead to an error (_itol undefined). I think the build is getting close. Also, I suspect that the typesetter c distribution from AT&T differed from what we have in terms of the build scripts and likely included binaries with the source: it seems that at some point ncc became necessary to build later verison of ncc. Thanks, Dan From jsg at jsg.id.au Thu Dec 5 19:26:17 2024 From: jsg at jsg.id.au (Jonathan Gray) Date: Thu, 5 Dec 2024 20:26:17 +1100 Subject: [TUHS] Amdahl UTS use by 386 designers Message-ID: I found it interesting that Amdahl UTS was used by designers of the 386. "we instituted Unix for the 386 design. Again if management knew what we were doing they wouldn't have let us do it." https://archive.computerhistory.org/resources/text/Oral_History/Intel_386_Design_and_Dev/102702019.05.01.acc.pdf pp. 14-15 Intel Alumni Network, 386 panel https://www.youtube.com/watch?v=fj8fnwuJGto&t=1540s Pat Gelsinger - UTS for Design Engineers https://www.gregbryant.com/intel/pat_gelsinger_unix_1984.pdf via https://mastodon.social/@aka_pugs/111723262047370983 "Given Pat was a bit of a UNIX hacker at the time, he set up the entire design team inside of his CMS account which was running the UTS UNIX environment from Amdahl." Gelsinger, Kirkpatrick, Kolodny, Singer Coping with the Complexity of Microprocessor Design at Intel - A CAD History https://kolodny.net.technion.ac.il/files/2016/07/IntelCADPaperFinal2.pdf From ron at ronnatalie.com Fri Dec 6 00:27:35 2024 From: ron at ronnatalie.com (Ron Natalie) Date: Thu, 05 Dec 2024 14:27:35 +0000 Subject: [TUHS] Porting AIX / Was Amdahl UTS use by 386 designers In-Reply-To: References: Message-ID: Years ago I worked on a project to port AIX to a couple of I860 microchannel add in cards. The first was the Intel “Wizard” card and the second was a four processor card with integral framebuffer coded the W4. We were running the AIX for the PS/2 on the host machine. AIX had a feature lifted from UCLA they called the Transparent Computing Facility. This allowed each executable to exist in the filesystem for multiple processor types. If you attempted to run an executable you didn’t have for your local machine, it would remote execute one of the ones you did have. This came in handy for the porting process. Amusingly, the i860 kernel I wrote bore more resemblance to the AIX/370 code than it did to the PS/2 (x86) code. We did have some laughs at IBM’s expense. The PS/2’s had this feature called the “High Function Terminal” that allowed you to switch the console between different virtual environments. We didn’t really have a console on the W4 so we referred to its virtual console as the “Low Function Terminal.” From crossd at gmail.com Fri Dec 6 01:19:15 2024 From: crossd at gmail.com (Dan Cross) Date: Thu, 5 Dec 2024 10:19:15 -0500 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: <20241205030843.8552FAB1EDA5@ary.qy> References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> Message-ID: On Wed, Dec 4, 2024 at 10:18 PM John Levine wrote: > It appears that Marc Donner said: > >With the notion of pipes it became possible to operate on data quickly and > >flexibly. There was nothing new from a fundamental capability point of > >view, but the ease with which one could construct pipelines enabled rapid > >experimentation and encouraged the development of pipe-able components to > >add to the tool set. > > Pipes were invented at least three times I'm aware of, but what made them > work so well in Unix is that they looked to the program the same as a file > so any program could use them for input or output without special arrangements, > and the shell made it easy to start two programs and pipe them together. Once you have coroutines and queues for passing data between them, a lot of things start to look like pipes. > The Dartmouth Time-Sharing System in the late 1960s had communication files > which were essentially two-way pipes, but they were asymmetrical. One end, the > slave end, looked like a file, but the other end, the master end, was different > and the program had to know it was a com file. They were mostly used to pass > terminal I/O between user programs at the slave end and SIMON at the master end, > the terminal monitor that talked to the front end computer than ran the TTYs. Doug has written at some length on this list about communication files and their homomorphism to the way Plan 9 presented and handled resources. > They were invented again at IBM in the 1970s and described in this paper. I wrote > them a letter, which they published, saying that Unix pipes did the same thing. > > https://dl.acm.org/doi/10.1147/sj.174.0383 Don't forget CMS pipelines, too! Sadly, the Morrison paper cited above is not easily accessible, though I acquired a copy from IEEE; perhaps a sucker's game as it was not cheap. Your subsequent letter, and part of Morrison's response to you, however, is available, gratis. Reading through Morrison, one gets the impression that there are some substantial differences with Unix pipelines; or rather, the way that Unix pipelines are usually used. In particular, he describes his linked streams in terms of a "network", by which he appears to mean an arbitrary directed graph. Crucially, he describes combining nodes, for merging data from multiple streams. Unix pipelines, on the other hand, tend to be used in a manner that is strictly linear, without the fan-out and fan-in capabilities described by Morrison. Of course, nothing prevents one from building a Morrison-style "network" from Unix processes and pipes, though it's hard to see how that would work without something like `select`, which didn't yet exist in 1978. Regardless, Unix still doesn't expose a particularly convenient syntax for expressing these sorts of constructions at the shell. As an aside, Morrison has a web page dedicated to "flow-based programming", that he claims to have invented in the late 1960s. That seems like a bit of a tall claim, and I'd wager Doug gives him a run for his money on that. https://jpaulm.github.io/fbp/ - Dan C. From johnl at taugh.com Fri Dec 6 02:00:40 2024 From: johnl at taugh.com (John R Levine) Date: 5 Dec 2024 11:00:40 -0500 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> Message-ID: <32bf4263-277d-8c8b-6bf7-e33df04a2c3c@taugh.com> On Thu, 5 Dec 2024, Dan Cross wrote: >> Pipes were invented at least three times I'm aware of, but what made them >> work so well in Unix is that they looked to the program the same as a file >> so any program could use them for input or output without special arrangements, >> and the shell made it easy to start two programs and pipe them together. > > Once you have coroutines and queues for passing data between them, a > lot of things start to look like pipes. They also can look a lot like temporary files. Someone, probably Heinz, did a shell for the tiny Unix that ran on floppies so this foo | bar actually did this foo > tmpfile ; bar < tmpfile; rm tmpfile to avoid having to swap programs in and out on floppies. The main disadvantage was that the tmpfile could overflow the tiny disks of the time. >> They were invented again at IBM in the 1970s and described in this paper. I wrote >> them a letter, which they published, saying that Unix pipes did the same thing. >> >> https://dl.acm.org/doi/10.1147/sj.174.0383 > > Don't forget CMS pipelines, too! > > Sadly, the Morrison paper cited above is not easily accessible, though If anyone else needs a copy, just ask. Regards, John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail. https://jl.ly From heinz at osta.com Fri Dec 6 02:17:01 2024 From: heinz at osta.com (Heinz Lycklama) Date: Thu, 5 Dec 2024 08:17:01 -0800 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: <32bf4263-277d-8c8b-6bf7-e33df04a2c3c@taugh.com> References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <32bf4263-277d-8c8b-6bf7-e33df04a2c3c@taugh.com> Message-ID: John, thanks for the reminder of the implementation of pipes on a constrained version of UNIX in the early days. The exact implementation is described on page 2095 of the BSTJ July-Aug 1978 for interested parties. Heinz On 12/5/2024 8:00 AM, John R Levine wrote: > On Thu, 5 Dec 2024, Dan Cross wrote: >>> Pipes were invented at least three times I'm aware of, but what made >>> them >>> work so well in Unix is that they looked to the program the same as >>> a file >>> so any program could use them for input or output without special >>> arrangements, >>> and the shell made it easy to start two programs and pipe them >>> together. >> >> Once you have coroutines and queues for passing data between them, a >> lot of things start to look like pipes. > > They also can look a lot like temporary files.  Someone, probably > Heinz, did a shell for the tiny Unix that ran on floppies so this > >  foo | bar > > actually did this > >  foo > tmpfile ; bar < tmpfile; rm tmpfile > > to avoid having to swap programs in and out on floppies.  The main > disadvantage was that the tmpfile could overflow the tiny disks of the > time. > >>> They were invented again at IBM in the 1970s and described in this >>> paper.  I wrote >>> them a letter, which they published, saying that Unix pipes did the >>> same thing. >>> >>> https://dl.acm.org/doi/10.1147/sj.174.0383 >> >> Don't forget CMS pipelines, too! >> >> Sadly, the Morrison paper cited above is not easily accessible, though > > If anyone else needs a copy, just ask. > > Regards, > John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY > Please consider the environment before reading this e-mail. https://jl.ly -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: RGxkomgHdfzyhAUv.png Type: image/png Size: 138780 bytes Desc: not available URL: From athornton at gmail.com Fri Dec 6 02:55:30 2024 From: athornton at gmail.com (Adam Thornton) Date: Thu, 5 Dec 2024 09:55:30 -0700 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> Message-ID: On Thu, Dec 5, 2024 at 8:20 AM Dan Cross wrote: > > Unix pipelines, on the other hand, tend to be used in a manner that is > strictly linear, without the fan-out and fan-in capabilities described > by Morrison. Of course, nothing prevents one from building a > Morrison-style "network" from Unix processes and pipes, though it's > hard to see how that would work without something like `select`, which > didn't yet exist in 1978. Regardless, Unix still doesn't expose a > particularly convenient syntax for expressing these sorts of > constructions at the shell. > > Rick Troth has recently published xfl, which is pretty much CMS Pipelines for Unix. https://github.com/trothtech/xfl He's got a slide deck on it at http://www.casita.net/pub/xfl/pervasive-vmws-2024.pdf . There are a lot of really cool things you can do with fanin/fanout. Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrochkind at gmail.com Fri Dec 6 03:06:47 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Thu, 5 Dec 2024 10:06:47 -0700 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <32bf4263-277d-8c8b-6bf7-e33df04a2c3c@taugh.com> Message-ID: I don't think files as pipes would be "transparent to the user." Reading an empty pipe causes a wait until the bytes requested are available, unless the pipe is closed first. Reading to the end of a file results in an end-of-file error. This problem is avoided if the source process completes before the target process begins, but then there is a different lack of transparency, which is that the processes don't run simultaneously. (I think this is the case with the implementation that Heinz showed.) Still, the same sort of pseudo-pipes were in MS-DOS, and they were occasionally useful. Marc On Thu, Dec 5, 2024 at 9:17 AM Heinz Lycklama wrote: > John, thanks for the reminder of the implementation > of pipes on a constrained version of UNIX in the early > days. The exact implementation is described on page 2095 > of the BSTJ July-Aug 1978 for interested parties. > > > > Heinz > > On 12/5/2024 8:00 AM, John R Levine wrote: > > On Thu, 5 Dec 2024, Dan Cross wrote: > > Pipes were invented at least three times I'm aware of, but what made them > work so well in Unix is that they looked to the program the same as a file > so any program could use them for input or output without special > arrangements, > and the shell made it easy to start two programs and pipe them together. > > > Once you have coroutines and queues for passing data between them, a > lot of things start to look like pipes. > > > They also can look a lot like temporary files. Someone, probably Heinz, > did a shell for the tiny Unix that ran on floppies so this > > foo | bar > > actually did this > > foo > tmpfile ; bar < tmpfile; rm tmpfile > > to avoid having to swap programs in and out on floppies. The main > disadvantage was that the tmpfile could overflow the tiny disks of the > time. > > They were invented again at IBM in the 1970s and described in this paper. > I wrote > them a letter, which they published, saying that Unix pipes did the same > thing. > > https://dl.acm.org/doi/10.1147/sj.174.0383 > > > Don't forget CMS pipelines, too! > > Sadly, the Morrison paper cited above is not easily accessible, though > > > If anyone else needs a copy, just ask. > > Regards, > John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY > Please consider the environment before reading this e-mail. https://jl.ly > > > -- *My new email address is mrochkind at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: RGxkomgHdfzyhAUv.png Type: image/png Size: 138780 bytes Desc: not available URL: From paul.winalski at gmail.com Fri Dec 6 03:22:05 2024 From: paul.winalski at gmail.com (Paul Winalski) Date: Thu, 5 Dec 2024 12:22:05 -0500 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <32bf4263-277d-8c8b-6bf7-e33df04a2c3c@taugh.com> Message-ID: Regarding pipes and pipe-like interprocess communications facilities in other operating systems, VMS has always had a pipe-like communications pseudo-devices called mailboxes. The main difference between Unix pipes and VMS mailboxes is that pipes have distinct read-only and write-only file descriptors. Mailboxes do not--channels (VMS-speak for file descriptors) assigned to a mailbox can be used for both reading and writing. This means that it is not possible to do "broken pipe"-type detection on a mailbox. This very much restricts the usefulness of mailboxes. I wrote a true pipe device driver for VMS as part of the DEC Shell product (a port of the Unix Bourne shell to VMS). -Paul W. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Fri Dec 6 03:35:31 2024 From: tuhs at tuhs.org (Chet Ramey via TUHS) Date: Thu, 5 Dec 2024 09:35:31 -0800 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> Message-ID: <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> On 12/5/24 10:19 AM, Dan Cross wrote: > Unix pipelines, on the other hand, tend to be used in a manner that is > strictly linear, without the fan-out and fan-in capabilities described > by Morrison. Of course, nothing prevents one from building a > Morrison-style "network" from Unix processes and pipes, though it's > hard to see how that would work without something like `select`, which > didn't yet exist in 1978. Regardless, Unix still doesn't expose a > particularly convenient syntax for expressing these sorts of > constructions at the shell. Process substitution is about as close as we can get, but most programs still process their filename arguments one at a time, beginning to end. The canonical process substitution example is diff <(old-program-version) <(new-program-version) to do simple regression testing. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature.asc Type: application/pgp-signature Size: 203 bytes Desc: OpenPGP digital signature URL: From cowan at ccil.org Fri Dec 6 03:53:06 2024 From: cowan at ccil.org (John Cowan) Date: Thu, 5 Dec 2024 12:53:06 -0500 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <32bf4263-277d-8c8b-6bf7-e33df04a2c3c@taugh.com> Message-ID: There was no concurrency in mini-Unix; the first pipeline component ran to completion, and when it exited the next component ran, and so on. In this way a pipeline could be arbitrarily long using just two disk files. On Thu, Dec 5, 2024, 12:07 PM Marc Rochkind wrote: > I don't think files as pipes would be "transparent to the user." Reading > an empty pipe causes a wait until the bytes requested are available, unless > the pipe is closed first. Reading to the end of a file results in an > end-of-file error. This problem is avoided if the source process completes > before the target process begins, but then there is a different lack of > transparency, which is that the processes don't run simultaneously. (I > think this is the case with the implementation that Heinz showed.) > > Still, the same sort of pseudo-pipes were in MS-DOS, and they were > occasionally useful. > > Marc > > On Thu, Dec 5, 2024 at 9:17 AM Heinz Lycklama wrote: > >> John, thanks for the reminder of the implementation >> of pipes on a constrained version of UNIX in the early >> days. The exact implementation is described on page 2095 >> of the BSTJ July-Aug 1978 for interested parties. >> >> >> >> Heinz >> >> On 12/5/2024 8:00 AM, John R Levine wrote: >> >> On Thu, 5 Dec 2024, Dan Cross wrote: >> >> Pipes were invented at least three times I'm aware of, but what made them >> work so well in Unix is that they looked to the program the same as a >> file >> so any program could use them for input or output without special >> arrangements, >> and the shell made it easy to start two programs and pipe them together. >> >> >> Once you have coroutines and queues for passing data between them, a >> lot of things start to look like pipes. >> >> >> They also can look a lot like temporary files. Someone, probably Heinz, >> did a shell for the tiny Unix that ran on floppies so this >> >> foo | bar >> >> actually did this >> >> foo > tmpfile ; bar < tmpfile; rm tmpfile >> >> to avoid having to swap programs in and out on floppies. The main >> disadvantage was that the tmpfile could overflow the tiny disks of the >> time. >> >> They were invented again at IBM in the 1970s and described in this >> paper. I wrote >> them a letter, which they published, saying that Unix pipes did the same >> thing. >> >> https://dl.acm.org/doi/10.1147/sj.174.0383 >> >> >> Don't forget CMS pipelines, too! >> >> Sadly, the Morrison paper cited above is not easily accessible, though >> >> >> If anyone else needs a copy, just ask. >> >> Regards, >> John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY >> Please consider the environment before reading this e-mail. https://jl.ly >> >> >> > > -- > *My new email address is mrochkind at gmail.com * > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: RGxkomgHdfzyhAUv.png Type: image/png Size: 138780 bytes Desc: not available URL: From johnl at taugh.com Fri Dec 6 04:05:41 2024 From: johnl at taugh.com (John Levine) Date: 5 Dec 2024 13:05:41 -0500 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <32bf4263-277d-8c8b-6bf7-e33df04a2c3c@taugh.com> Message-ID: <20241205180541.E2818AB56024@ary.qy> It appears that Marc Rochkind said: >-=-=-=-=-=- >-=-=-=-=-=- > >I don't think files as pipes would be "transparent to the user." Reading an >empty pipe causes a wait until the bytes requested are available, unless >the pipe is closed first. ... You should read Meinz' paper. This was a cut down version of Unix that ran in 40K bytes on an LSI-11 with a couple of floppy disks. It ran one program at a time, since swapping to floppies was absurdly slow. The pseudo-pipe ran the first program which wrote the output to a file, then ran the second program which read it. It was close enough for most purposes so long as the disk didn't fill up. >Still, the same sort of pseudo-pipes were in MS-DOS, and they were >occasionally useful. I can think of lots of places that had a way for one program to write a temporary file that a subsequent one read. For example, in OS/360 JCL you'd write (well, punch) something like this for the object output of the Fortran compiler //SYSLIN DD DSNAME=&OBJECT,DISP=(NEW,PASS),UNIT=SYSSQ and then in the linker read it in //SYSIN DD DSNAME=&OBJECT,DISP=(OLD,DELETE) The & in the name said it was a temp file so make up a unique name. I probably didn't get the JCL exactly right since it's been about 50 years since I wrote any. R's, John From ron at ronnatalie.com Fri Dec 6 04:19:16 2024 From: ron at ronnatalie.com (Ron Natalie) Date: Thu, 05 Dec 2024 18:19:16 +0000 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: <32bf4263-277d-8c8b-6bf7-e33df04a2c3c@taugh.com> References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <32bf4263-277d-8c8b-6bf7-e33df04a2c3c@taugh.com> Message-ID: I remember on MiniUnix (a stripped down kernel that would run on PDP11’s without memory management, in our case 11/03, 11/20, and an older 11/40) there were no kernel pipes (and limited process concurrency). The shell used the aforementioned emulation of pipe by just running the output of one process into a temporary file subsequently loaded as stdin of the other. We kind of put all that to bed when the 11/23s came out and we could run our full up kernel on the micros. From cmhanson at eschatologist.net Fri Dec 6 05:23:01 2024 From: cmhanson at eschatologist.net (Chris Hanson) Date: Thu, 5 Dec 2024 11:23:01 -0800 Subject: [TUHS] I threw together a MINIX 1.5 system call emulator Message-ID: <51ACDEF8-362E-46AB-9B24-39A7FB273D57@eschatologist.net> Since MINIX was a UNIX V7 clone for teaching, I figure this is at least somewhat on-topic. I’ve wanted to port MINIX 1.5 for M68000 to other systems besides Amiga, Atari ST, and the classic Mac, but trying to do that within a system emulator is a pain and doesn’t help you use a modern editor or SCM system. So I took the Musashi M68000 emulator and, using the MINIX 1.5 sources for Atari ST for reference, I’ve implemented a system call emulator that’s now _almost_ sufficient to run /usr/bin/cc. It’s up on GitHub at https://github.com/eschaton/MINIXCompat and I’ve released it under an MIT license. It requires my forked version of the Musashi project that supports implementing a TRAP instruction via a callback, which is necessary for implementing system calls on the host side. I reference this via a submodule so it can be kept at least logically distinct from the rest of the code. There’s no Makefile as I’m using Xcode on macOS to develop it, though I expect to write one at some point so I can run it on NetBSD and Linux as well as on macOS; writing one should be straightforward. -- Chris From arnold at skeeve.com Fri Dec 6 06:55:34 2024 From: arnold at skeeve.com (arnold at skeeve.com) Date: Thu, 05 Dec 2024 13:55:34 -0700 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> Message-ID: <202412052055.4B5KtYf1781856@freefriends.org> Chet Ramey via TUHS wrote: > On 12/5/24 10:19 AM, Dan Cross wrote: > > > Unix pipelines, on the other hand, tend to be used in a manner that is > > strictly linear, without the fan-out and fan-in capabilities described > > by Morrison. Of course, nothing prevents one from building a > > Morrison-style "network" from Unix processes and pipes, though it's > > hard to see how that would work without something like `select`, which > > didn't yet exist in 1978. Regardless, Unix still doesn't expose a > > particularly convenient syntax for expressing these sorts of > > constructions at the shell. > > Process substitution is about as close as we can get, but most programs > still process their filename arguments one at a time, beginning to end. > > The canonical process substitution example is > > diff <(old-program-version) <(new-program-version) > > to do simple regression testing. And fanout is simply ... | tee >(pipeline1) >(pipeline2) Arnold From crossd at gmail.com Fri Dec 6 07:12:34 2024 From: crossd at gmail.com (Dan Cross) Date: Thu, 5 Dec 2024 16:12:34 -0500 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: <202412052055.4B5KtYf1781856@freefriends.org> References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> Message-ID: On Thu, Dec 5, 2024 at 3:56 PM wrote: > Chet Ramey via TUHS wrote: > > On 12/5/24 10:19 AM, Dan Cross wrote: > > > > > Unix pipelines, on the other hand, tend to be used in a manner that is > > > strictly linear, without the fan-out and fan-in capabilities described > > > by Morrison. Of course, nothing prevents one from building a > > > Morrison-style "network" from Unix processes and pipes, though it's > > > hard to see how that would work without something like `select`, which > > > didn't yet exist in 1978. Regardless, Unix still doesn't expose a > > > particularly convenient syntax for expressing these sorts of > > > constructions at the shell. > > > > Process substitution is about as close as we can get, but most programs > > still process their filename arguments one at a time, beginning to end. > > > > The canonical process substitution example is > > > > diff <(old-program-version) <(new-program-version) > > > > to do simple regression testing. > > And fanout is simply > > ... | tee >(pipeline1) >(pipeline2) And indeed these things are pretty nifty, but don't they generate trees, and not arbitrary dags? They don't quite capture the full generality of Morrison-style networks since it doesn't seem like there's a way to connect process substitution fan-out with fan-in; at least, not conveniently. It is, perhaps, notable that Go allows me to do this sort of thing with channels and goroutines, but it has `select` built into the language. Funny, despite using Unix almost daily for over 30 years now, I don't think I've ever felt limited by the power of pipelines. On the contrary, I've lost count of the times I've felt limited on systems that do Not support pipes. - Dan C. From mrochkind at gmail.com Fri Dec 6 07:50:56 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Thu, 5 Dec 2024 14:50:56 -0700 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> Message-ID: Some of the posts in this thread confuse pipes in UNIX shells (the | symbol) with the pipe system call. In the shell, how two processes can be connected with a pipe is very constrained (only one unidirectional pipe). But the pipe system call can be used to build much more elaborate connections. Back in 1980, when I was still at Bell Labs, I wrote a shell called 2dsh ("two dimensional shell) that had a more complex syntax. The memo I wrote, "2DSH—An experimental shell for connecting processes with multiple data streams", wasn't published externally, but exists as a Bell Labs memo. I found a reference here: https://scholar.google.com/scholar_lookup?title=2DSH%E2%80%94An+experimental+shell+for+connecting+processes+with+multiple+data+streams&author=M.+J.+Rochkind&publication_year=1980 . Here are two examples from that memo: [image: image.png] I stumbled across another paper from 2017 titled "Extending Unix Pipelines to DAGs," which references my un-published Bell Labs memo. I haven't read it since I don't subscribe to IEEE Transactions on Computers. A while ago Doug McIlroy was kind enough to send me a scan of my memo, but I don't think I'm allowed to publish it here. In that memo, I credit Doug for coming with a very similar idea around the same time ("A Notation for Arboreal Plumbing"). Marc Rochkind On Thu, Dec 5, 2024 at 2:13 PM Dan Cross wrote: > On Thu, Dec 5, 2024 at 3:56 PM wrote: > > Chet Ramey via TUHS wrote: > > > On 12/5/24 10:19 AM, Dan Cross wrote: > > > > > > > Unix pipelines, on the other hand, tend to be used in a manner that > is > > > > strictly linear, without the fan-out and fan-in capabilities > described > > > > by Morrison. Of course, nothing prevents one from building a > > > > Morrison-style "network" from Unix processes and pipes, though it's > > > > hard to see how that would work without something like `select`, > which > > > > didn't yet exist in 1978. Regardless, Unix still doesn't expose a > > > > particularly convenient syntax for expressing these sorts of > > > > constructions at the shell. > > > > > > Process substitution is about as close as we can get, but most programs > > > still process their filename arguments one at a time, beginning to end. > > > > > > The canonical process substitution example is > > > > > > diff <(old-program-version) <(new-program-version) > > > > > > to do simple regression testing. > > > > And fanout is simply > > > > ... | tee >(pipeline1) >(pipeline2) > > And indeed these things are pretty nifty, but don't they generate > trees, and not arbitrary dags? They don't quite capture the full > generality of Morrison-style networks since it doesn't seem like > there's a way to connect process substitution fan-out with fan-in; at > least, not conveniently. > > It is, perhaps, notable that Go allows me to do this sort of thing > with channels and goroutines, but it has `select` built into the > language. > > Funny, despite using Unix almost daily for over 30 years now, I don't > think I've ever felt limited by the power of pipelines. On the > contrary, I've lost count of the times I've felt limited on systems > that do Not support pipes. > > - Dan C. > -- *My new email address is mrochkind at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 368482 bytes Desc: not available URL: From imp at bsdimp.com Fri Dec 6 08:03:18 2024 From: imp at bsdimp.com (Warner Losh) Date: Thu, 5 Dec 2024 15:03:18 -0700 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> Message-ID: On Thu, Dec 5, 2024 at 2:13 PM Dan Cross wrote: > On Thu, Dec 5, 2024 at 3:56 PM wrote: > > Chet Ramey via TUHS wrote: > > > On 12/5/24 10:19 AM, Dan Cross wrote: > > > > > > > Unix pipelines, on the other hand, tend to be used in a manner that > is > > > > strictly linear, without the fan-out and fan-in capabilities > described > > > > by Morrison. Of course, nothing prevents one from building a > > > > Morrison-style "network" from Unix processes and pipes, though it's > > > > hard to see how that would work without something like `select`, > which > > > > didn't yet exist in 1978. Regardless, Unix still doesn't expose a > > > > particularly convenient syntax for expressing these sorts of > > > > constructions at the shell. > > > > > > Process substitution is about as close as we can get, but most programs > > > still process their filename arguments one at a time, beginning to end. > > > > > > The canonical process substitution example is > > > > > > diff <(old-program-version) <(new-program-version) > > > > > > to do simple regression testing. > > > > And fanout is simply > > > > ... | tee >(pipeline1) >(pipeline2) > > And indeed these things are pretty nifty, but don't they generate > trees, and not arbitrary dags? They don't quite capture the full > generality of Morrison-style networks since it doesn't seem like > there's a way to connect process substitution fan-out with fan-in; at > least, not conveniently. > > It is, perhaps, notable that Go allows me to do this sort of thing > with channels and goroutines, but it has `select` built into the > language. > > Funny, despite using Unix almost daily for over 30 years now, I don't > think I've ever felt limited by the power of pipelines. On the > contrary, I've lost count of the times I've felt limited on systems > that do Not support pipes. > The <() , >() syntax is a bash extension. Not all shells support it. And I couldn't find them in POSIX Issue 8. Warner -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Fri Dec 6 08:05:26 2024 From: tuhs at tuhs.org (Bakul Shah via TUHS) Date: Thu, 5 Dec 2024 14:05:26 -0800 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: Message-ID: <19B4DBDC-4342-4557-925D-8D546DC9D84F@iitbombay.org> On Dec 5, 2024, at 1:13 PM, Dan Cross wrote: > > And indeed these things are pretty nifty, but don't they generate > trees, and not arbitrary dags? They don't quite capture the full > generality of Morrison-style networks since it doesn't seem like > there's a way to connect process substitution fan-out with fan-in; at > least, not conveniently. Isn’t that a limitation of the shell language(s)? It would be easier in a 2D graphical shell but that would perhaps make specifying the more common simpler pipelines harder! A "linear" syntax would require *naming*. For example, let-proc a = "foo -a", b = "bar -c -d", c = "zee" in a.in1 = file1, a.in2 = file2, b.in1 = a.out, b.in2 = file3, c.in1 = b.out1, c.in2 = a.out, c.out = file4; Here names a, b & c are used to denote *processes*, with connections specified in the next two lines. Once all the pipes are created, processes a, b & c can be started giving them the right endpoints in forked processes prior to execing. Doable but not really worth it. At this point you may as well use s-expressions! From tuhs at tuhs.org Fri Dec 6 08:19:43 2024 From: tuhs at tuhs.org (Chet Ramey via TUHS) Date: Thu, 5 Dec 2024 14:19:43 -0800 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> Message-ID: <002cf6d3-0ed2-46f2-9fae-19a3e10d7b9c@case.edu> On 12/5/24 5:03 PM, Warner Losh wrote: > The <() , >()  syntax is a bash extension. Not all shells support it. And > I couldn't find them in POSIX Issue 8. Credit where credit is due: I picked them up from ksh93, and extended them to use named pipes on systems where /dev/fd isn't available. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature.asc Type: application/pgp-signature Size: 203 bytes Desc: OpenPGP digital signature URL: From mrochkind at gmail.com Fri Dec 6 09:07:14 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Thu, 5 Dec 2024 16:07:14 -0700 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: <002cf6d3-0ed2-46f2-9fae-19a3e10d7b9c@case.edu> References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> <002cf6d3-0ed2-46f2-9fae-19a3e10d7b9c@case.edu> Message-ID: I found that 2017 paper "Extending Unix Pipelines to DAGs". It's open access: https://ieeexplore.ieee.org/document/7903579 The open source code itself is here: https://github.com/dspinellis/dgsh Maybe an ambitious TUHS contributor can get the code running and give us a report. Marc Rochkind On Thu, Dec 5, 2024 at 3:38 PM Chet Ramey via TUHS wrote: > On 12/5/24 5:03 PM, Warner Losh wrote: > > > The <() , >() syntax is a bash extension. Not all shells support it. And > > I couldn't find them in POSIX Issue 8. > > Credit where credit is due: I picked them up from ksh93, and extended > them to use named pipes on systems where /dev/fd isn't available. > > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ > -- *My new email address is mrochkind at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Fri Dec 6 09:07:10 2024 From: arnold at skeeve.com (arnold at skeeve.com) Date: Thu, 05 Dec 2024 16:07:10 -0700 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> Message-ID: <202412052307.4B5N7Ade790095@freefriends.org> Warner Losh wrote: > The <() , >() syntax is a bash extension. Not all shells support it. And > I couldn't find them in POSIX Issue 8. > > Warner It originated in ksh93. 30 years have gone by, it's time it was added to POSIX. At least, IMHO. Thanks, Arnold From flexibeast at gmail.com Fri Dec 6 10:46:43 2024 From: flexibeast at gmail.com (Alexis) Date: Fri, 06 Dec 2024 11:46:43 +1100 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: <002cf6d3-0ed2-46f2-9fae-19a3e10d7b9c@case.edu> (Chet Ramey via TUHS's message of "Thu, 5 Dec 2024 14:19:43 -0800") References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> <002cf6d3-0ed2-46f2-9fae-19a3e10d7b9c@case.edu> Message-ID: <87ttbhzlho.fsf@gmail.com> Chet Ramey via TUHS writes: > On 12/5/24 5:03 PM, Warner Losh wrote: > >> The <() , >()  syntax is a bash extension. Not all shells >> support >> it. And >> I couldn't find them in POSIX Issue 8. > > Credit where credit is due: I picked them up from ksh93, and > extended > them to use named pipes on systems where /dev/fd isn't > available. i must say i was moderately surprised to learn a few years ago that process substitution wasn't in POSIX - i find it succinct syntactic sugar. There's a post on Chris Siebenmann's blog about some of the issues involved, including the /dev/fd issue: https://utcc.utoronto.ca/~cks/space/blog/unix/ProcessSubstitutionWhyLate Can anyone point me to any discussions about process substitution potentially becoming part of POSIX? This search on the Austin Group tracker didn't yield any useful results: https://austingroupbugs.net/search.php?project_id=0&search=process+substitution&sticky_issues=off&sortby=last_updated&dir=DESC&hide_status_id=-2&FILTER_SEARCH_PLATFORM= Alexis. From g.branden.robinson at gmail.com Fri Dec 6 11:09:26 2024 From: g.branden.robinson at gmail.com (G. Branden Robinson) Date: Thu, 5 Dec 2024 19:09:26 -0600 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: <202412052307.4B5N7Ade790095@freefriends.org> References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> <202412052307.4B5N7Ade790095@freefriends.org> Message-ID: <20241206010926.btnkeeqx7hhnoh47@illithid> At 2024-12-05T16:07:10-0700, arnold at skeeve.com wrote: > Warner Losh wrote: > > The <() , >() syntax is a bash extension. Not all shells support > > it. And I couldn't find them in POSIX Issue 8. > > It originated in ksh93. Are you sure? I think Tom Duff originated it in his "rc" shell. https://archive.org/details/rc-shell Regards, Branden -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From woods at robohack.ca Fri Dec 6 11:31:15 2024 From: woods at robohack.ca (Greg A. Woods) Date: Thu, 05 Dec 2024 17:31:15 -0800 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: <20241206010926.btnkeeqx7hhnoh47@illithid> References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> <202412052307.4B5N7Ade790095@freefriends.org> <20241206010926.btnkeeqx7hhnoh47@illithid> Message-ID: At Thu, 5 Dec 2024 19:09:26 -0600, "G. Branden Robinson" wrote: Subject: [TUHS] Re: Pipes (was Re: After 50 years, what has the Impact of Unix been?) > > [1 ] > At 2024-12-05T16:07:10-0700, arnold at skeeve.com wrote: > > Warner Losh wrote: > > > The <() , >() syntax is a bash extension. Not all shells support > > > it. And I couldn't find them in POSIX Issue 8. > > > > It originated in ksh93. > > Are you sure? I think Tom Duff originated it in his "rc" shell. The Wikipedia entry for "Process substitution" says: Process substitution was available as a compile-time option for ksh88, the 1988 version of the KornShell from Bell Labs. The rc shell provides the feature as "pipeline branching" in Version 10 Unix, released in 1990. So, if we believe that then indeed ksh88 pioneered process substitution. -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From douglas.mcilroy at dartmouth.edu Fri Dec 6 11:36:15 2024 From: douglas.mcilroy at dartmouth.edu (Douglas McIlroy) Date: Thu, 5 Dec 2024 20:36:15 -0500 Subject: [TUHS] After 50 years, what has the Impact of Unix been? Message-ID: > Pipes were invented at least three times I would add a 4th: POGOL--a remarkable data-processing language from NSA, described by Gloria Lambert at the first POPL conference, 1973. A POGOL program is made of processes that read and write notional files. The compiler considers all the processes together and optimizes files out of existence wherever it sees that written data can be read immediately. Otherwise a buffering file is instantiated. Unlike Unix pipes, though, a pair of communicating processes must share common knowledge about the connection--the file name. A ready-made theory of pipe networks was published essentially simultaneously with the advent of DTSS communication files: R.M. Karp, R.E. Miller, and S. Winograd. The organization of computations for uniform recurrence equations. Journal of the ACM, 14(3):563-590, July 1967. Completely unsuspecting processes could have been connected by a pair of DTSS communication files controlled by a master relay process. As far as I know this was never done, although such a mechanism was used for the DTSS chat facility. For the special case of clocked sample-data networks, BLODI (block diagram compiler) by Lochbaum, Kelly and Vyssotsky was way ahead (1960) of all the pipe-based formalisms. Doug From johnl at taugh.com Fri Dec 6 12:02:24 2024 From: johnl at taugh.com (John Levine) Date: 5 Dec 2024 21:02:24 -0500 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <20241205030843.8552FAB1EDA5@ary.qy> <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <202412052055.4B5KtYf1781856@freefriends.org> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> Message-ID: <20241206020224.E59B3AB6EF07@ary.qy> According to Dan Cross : >> > diff <(old-program-version) <(new-program-version) >> > >> > to do simple regression testing. >> >> And fanout is simply >> >> ... | tee >(pipeline1) >(pipeline2) > >And indeed these things are pretty nifty, but don't they generate >trees, and not arbitrary dags? They don't quite capture the full >generality of Morrison-style networks since it doesn't seem like >there's a way to connect process substitution fan-out with fan-in; at >least, not conveniently. You can use mkfifo to make named pipes and then plumb them as wanted. It's not particularly beautiful but it's as general as you could want. That's worked since 4.4BSD which was quite a while ago. From usotsuki at buric.co Fri Dec 6 12:05:51 2024 From: usotsuki at buric.co (Steve Nickolas) Date: Thu, 5 Dec 2024 21:05:51 -0500 (EST) Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> <202412052307.4B5N7Ade790095@freefriends.org> <20241206010926.btnkeeqx7hhnoh47@illithid> Message-ID: On Thu, 5 Dec 2024, Greg A. Woods wrote: > The Wikipedia entry for "Process substitution" says: > > Process substitution was available as a compile-time option for > ksh88, the 1988 version of the KornShell from Bell Labs. The rc > shell provides the feature as "pipeline branching" in Version 10 > Unix, released in 1990. > > So, if we believe that then indeed ksh88 pioneered process substitution. Confirming that ksh88d and ksh88i support <(command) at least. -uso. From crossd at gmail.com Fri Dec 6 12:21:49 2024 From: crossd at gmail.com (Dan Cross) Date: Thu, 5 Dec 2024 21:21:49 -0500 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: <20241206020224.E59B3AB6EF07@ary.qy> References: <20241205030843.8552FAB1EDA5@ary.qy> <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <202412052055.4B5KtYf1781856@freefriends.org> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <20241206020224.E59B3AB6EF07@ary.qy> Message-ID: On Thu, Dec 5, 2024 at 9:02 PM John Levine wrote: > According to Dan Cross : > >> > diff <(old-program-version) <(new-program-version) > >> > > >> > to do simple regression testing. > >> > >> And fanout is simply > >> > >> ... | tee >(pipeline1) >(pipeline2) > > > >And indeed these things are pretty nifty, but don't they generate > >trees, and not arbitrary dags? They don't quite capture the full > >generality of Morrison-style networks since it doesn't seem like > >there's a way to connect process substitution fan-out with fan-in; at > >least, not conveniently. > > You can use mkfifo to make named pipes and then plumb them as wanted. It's > not particularly beautiful but it's as general as you could want. > > That's worked since 4.4BSD which was quite a while ago. Sure, but that's orthogonal since it's not really using shell syntax to build the thing up in a canonical way. I mean, perhaps in some literal sense it is (one types the commands into the interpreter...) but nowhere near the expressiveness of `ls | grep foo | whatever`. Fundamentally, the questions here are all about syntax. As I wrote in my first email, the system calls are very general, and could be used to build arbitrary directed graphs (not even limited to DAGs) of processes connected by pipes in all sorts of configurations (presumably those processes would be written to handle multiple input and output streams), without having to resort to named pipes. One can imagine some sort of DSL that describes such graphs of pipes and processes (Bakul alluded to this), and an interpreter for that DSL that creates the described set of pipes and processes, all appropriately connected and running. One could even imagine such a thing being used as part of a more traditional pipeline. My claim is simply that extant shell syntax isn't really amenable to this in a natural way. As Chet and Arnold pointed out, process substitution as pioneered in ksh gets us a little closer, but it's not quite as general (I believe any such graphs would have to be acyclic); it's certainly not as syntactically pleasant. There has been work along these lines; I was sent a reference off-list to a paper by Spinellis and Fragkoulis about a DAG-oriented shell: https://www.spinellis.gr/sw/dgsh/, which seems relevant. - Dan C. From douglas.mcilroy at dartmouth.edu Fri Dec 6 12:26:48 2024 From: douglas.mcilroy at dartmouth.edu (Douglas McIlroy) Date: Thu, 5 Dec 2024 21:26:48 -0500 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) Message-ID: Many tree and dag pipe notations have been proposed. Marc's was one of the first. I devised one myself, but didn't like it enough to publish it even as a TM. A recent one is Spinellis's dgsh. More elaborate networks with feedback loops evolve for power-series computation. The idea of implementing cellular automata as arrays of processes with nearest neighbors connected by pipes was suggested early on, but I've never seen such an arrangement implemented--it would be hideously slow. I once wrote a general plumber that worked from a connection list: connect fd m in process A to fd n in process B. The main challenge was to find an order of hooking things up so that the number of live file descriptors in the plumber didn't exceed the system limit. Doug From athornton at gmail.com Fri Dec 6 12:29:24 2024 From: athornton at gmail.com (Adam Thornton) Date: Thu, 5 Dec 2024 19:29:24 -0700 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <32bf4263-277d-8c8b-6bf7-e33df04a2c3c@taugh.com> Message-ID: /23+, no? The basic 23 doesn't have an MMU, I'm pretty sure. On Thu, Dec 5, 2024 at 11:58 AM Ron Natalie wrote: > I remember on MiniUnix (a stripped down kernel that would run on PDP11’s > without memory management, in our case 11/03, 11/20, and an older 11/40) > there were no kernel pipes (and limited process concurrency). > The shell used the aforementioned emulation of pipe by just running the > output of one process into a temporary file subsequently loaded as stdin > of the other. > > We kind of put all that to bed when the 11/23s came out and we could run > our full up kernel on the micros. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dds at aueb.gr Fri Dec 6 18:16:38 2024 From: dds at aueb.gr (Diomidis Spinellis) Date: Fri, 6 Dec 2024 10:16:38 +0200 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> <002cf6d3-0ed2-46f2-9fae-19a3e10d7b9c@case.edu> Message-ID: <4c107309-75b7-42b3-ada3-161ec1421bd5@aueb.gr> On 06-Dec-24 01:07, Marc Rochkind wrote: > I found that 2017 paper "Extending Unix Pipelines to DAGs". It's open > access: > > https://ieeexplore.ieee.org/document/7903579 ieeexplore.ieee.org/document/7903579> > > The open source code itself is here: https://github.com/dspinellis/dgsh > > > Maybe an ambitious TUHS contributor can get the code running and give us > a report. I wrote the dgsh code with my co-author Marios Fragkoulis, so I still have it running. Doug McIlroy, who also mentioned dgsh in another message, is too modest to say that its design owes much to his input. I asked him for feedback when I was working on it, and over several iterations he proposed important (and quite demanding as I recall) improvements to its design. The system allows the concise and readable expression of several graph topologies I had in mind when I started working on it, and more [1]. However, it hasn't caught on. I think the main reason is that it is based on modified versions of several existing tools (bash, cmp, comm, cut, diff, diff3, grep, join, paste, perm, sort) [2]. The modifications allow the tools to coordinate between them the setup of pipes when placed in a dgsh graph according to available inputs and required outputs. The changes (especially for bash) aren't small, which meant that I didn't think it was realistic to push them upstream, which now means that the modified tools are out of date and difficult to build. Not sure what can be done to address this problem. It seems that a widely adopted system, such as modern Unix/Linux, has too much inertia for it to adopt potentially disrupting innovations. In retrospect, the way we designed the pipe graph setup could also be improved. The current design involves an initial phase where IPC messages are circulating around the graph to communicate the I/O requirements of each tool, for example that comm(1) should expect input from two processes and output to three processes. The design is brittle and difficult to troubleshoot, because coordination happens dynamically behind the scenes. A better design (and one I think Doug was advocating) would statically analyze the graph's topology and invoke each tool with appropriate parameters or environment variables. However, this design would require significantly more extensive modifications to bash, or the implementation of a new shell. Both approaches required work for which we didn't have the time and energy at the time, and also had their own downsides regarding adoption potential. [1] https://www.spinellis.gr/sw/dgsh/#examples [2] https://www.spinellis.gr/sw/dgsh/#tools Diomidis From mrochkind at gmail.com Sat Dec 7 02:10:18 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Fri, 6 Dec 2024 09:10:18 -0700 Subject: [TUHS] Interesting post about Microsoft and UNIX Message-ID: I just came across a 1995 post from Gordon Letwin, early Microsoft employee and lead architect of OS/2, about the history of OS/2. There are a few paragraphs in it about Microsoft and UNIX. Here's Letwin's post: https://gunkies.org/wiki/Gordon_Letwin_OS/2_usenet_post And the UNIX-related paragraphs: *It's extremely hard to do development work on an operating system when someone else controls the standard. "Control" in this case is a matter of public perception. For example, Microsoft was once very big in the Unix world. In fact, we considered it our candidate for the future desktop operating system, when machines got powerful enough to run something good. We were the worlds biggest seller of Unix systems. DOS was, when we first wrote it, a one-time throw-away product intended to keep IBM happy so that they'd buy our languages.The UNIX contracts were all done when Bell Labs was regulated and couldn't sell Unix into the commerical marketplace. So although they wrote it and were paid royalties, they couldn't develop it in competition to us. But after a few years that changed. Bell was degregulated and now they were selling Unix directly, in competition to us! They might sell it for cheaper than we had to pay them in royalties! But that wasn't the real killer, the real killer was the Bell now controlled the standard. If we wrote an API extension that did X, and Bell wrote an incompatible one that did Y, which one would people write for? The ISVs know that AT&T was a very big company and that they'd written the original, so they'd believe that AT&T controlled the standard, not MS, and that belief would then define reality. So we'd always just be waiting for what AT&T announced and then frantically trying to duplicate it.Bill Gates knew, right away, that there was no strong future in Unix for us any more. Fortunately at that time, DOS was taking off and we were learning, along with everyone else, about the power of standards. So the primary OS team - the Unix guys - joined with the secondary OS team - the DOS guys - and the earliest versions of OS/2 were born. (This was before IBM came on board, so it wasn't called OS/2!)* Marc Rochkind -------------- next part -------------- An HTML attachment was scrubbed... URL: From krewat at kilonet.net Sat Dec 7 02:38:28 2024 From: krewat at kilonet.net (Arthur Krewat) Date: Fri, 6 Dec 2024 16:38:28 +0000 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: Message-ID: And by concatenation, that's how we wound up with a VMS clone on our desktops. ________________________________ From: Marc Rochkind Sent: Friday, December 6, 2024 11:10 AM To: The UNIX Historical Society Subject: [TUHS] Interesting post about Microsoft and UNIX I just came across a 1995 post from Gordon Letwin, early Microsoft employee and lead architect of OS/2, about the history of OS/2. There are a few paragraphs in it about Microsoft and UNIX. Here's Letwin's post: https://gunkies.org/wiki/Gordon_Letwin_OS/2_usenet_post And the UNIX-related paragraphs: It's extremely hard to do development work on an operating system when someone else controls the standard. "Control" in this case is a matter of public perception. For example, Microsoft was once very big in the Unix world. In fact, we considered it our candidate for the future desktop operating system, when machines got powerful enough to run something good. We were the worlds biggest seller of Unix systems. DOS was, when we first wrote it, a one-time throw-away product intended to keep IBM happy so that they'd buy our languages. The UNIX contracts were all done when Bell Labs was regulated and couldn't sell Unix into the commerical marketplace. So although they wrote it and were paid royalties, they couldn't develop it in competition to us. But after a few years that changed. Bell was degregulated and now they were selling Unix directly, in competition to us! They might sell it for cheaper than we had to pay them in royalties! But that wasn't the real killer, the real killer was the Bell now controlled the standard. If we wrote an API extension that did X, and Bell wrote an incompatible one that did Y, which one would people write for? The ISVs know that AT&T was a very big company and that they'd written the original, so they'd believe that AT&T controlled the standard, not MS, and that belief would then define reality. So we'd always just be waiting for what AT&T announced and then frantically trying to duplicate it. Bill Gates knew, right away, that there was no strong future in Unix for us any more. Fortunately at that time, DOS was taking off and we were learning, along with everyone else, about the power of standards. So the primary OS team - the Unix guys - joined with the secondary OS team - the DOS guys - and the earliest versions of OS/2 were born. (This was before IBM came on board, so it wasn't called OS/2!) Marc Rochkind -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Sat Dec 7 02:44:06 2024 From: arnold at skeeve.com (arnold at skeeve.com) Date: Fri, 06 Dec 2024 09:44:06 -0700 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> <202412052307.4B5N7Ade790095@freefriends.org> <20241206010926.btnkeeqx7hhnoh47@illithid> Message-ID: <202412061644.4B6Gi6Ok871437@freefriends.org> "Greg A. Woods" wrote: > > Are you sure? I think Tom Duff originated it in his "rc" shell. > > The Wikipedia entry for "Process substitution" says: > > Process substitution was available as a compile-time option for > ksh88, the 1988 version of the KornShell from Bell Labs. The rc > shell provides the feature as "pipeline branching" in Version 10 > Unix, released in 1990. > > So, if we believe that then indeed ksh88 pioneered process substitution. I'd forgotten that it was optional in ksh88. It was there out of the box in ksh93. Arnold From arnold at skeeve.com Sat Dec 7 02:46:31 2024 From: arnold at skeeve.com (arnold at skeeve.com) Date: Fri, 06 Dec 2024 09:46:31 -0700 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <20241205030843.8552FAB1EDA5@ary.qy> <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <202412052055.4B5KtYf1781856@freefriends.org> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <20241206020224.E59B3AB6EF07@ary.qy> Message-ID: <202412061646.4B6GkVgU871634@freefriends.org> Dan Cross wrote: > My claim is simply that extant shell syntax isn't really amenable to > this in a natural way. As Chet and Arnold pointed out, process > substitution as pioneered in ksh gets us a little closer, but it's not > quite as general (I believe any such graphs would have to be acyclic); > it's certainly not as syntactically pleasant. > > There has been work along these lines; I was sent a reference off-list > to a paper by Spinellis and Fragkoulis about a DAG-oriented shell: > https://www.spinellis.gr/sw/dgsh/, which seems relevant. In the early-to-mid 1980s, the Georgia Tech Software Tools Subsystem for Prime Computers offered three standard inputs, three standard outputs, and a shell with syntax to connect things as you desired. I don't remember the syntax though. I also don't know how much this feature was really used. Arnold From aek at bitsavers.org Sat Dec 7 03:05:29 2024 From: aek at bitsavers.org (Al Kossow) Date: Fri, 6 Dec 2024 09:05:29 -0800 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: Message-ID: On 12/6/24 8:38 AM, Arthur Krewat wrote: > /his was before IBM came on board/ and IBM had no interest in the 386 at the time, which set the wheels in motion for Gates to acquire Cutler and DECwest From tuhs at tuhs.org Sat Dec 7 03:28:57 2024 From: tuhs at tuhs.org (segaloco via TUHS) Date: Fri, 06 Dec 2024 17:28:57 +0000 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: Message-ID: <26vRiRS2sO-BAumb5V9ClzUqNJTMHE6ISl04n95AGMV2z3Kx_mmG9qOIIsLMGAAWJKprznaC9rGeI7_bX842X__q39Ind8b5O0JfuSkoLkk=@protonmail.com> On Friday, December 6th, 2024 at 8:10 AM, Marc Rochkind wrote: > I just came across a 1995 post from Gordon Letwin, early Microsoft employee and lead architect of OS/2, about the history of OS/2. There are a few paragraphs in it about Microsoft and UNIX. Here's Letwin's post: > > https://gunkies.org/wiki/Gordon_Letwin_OS/2_usenet_post > > And the UNIX-related paragraphs: > > It's extremely hard to do development work on an operating system when someone else controls the standard. "Control" in this case is a matter of public perception. For example, Microsoft was once very big in the Unix world. In fact, we considered it our candidate for the future desktop operating system, when machines got powerful enough to run something good. We were the worlds biggest seller of Unix systems. DOS was, when we first wrote it, a one-time throw-away product intended to keep IBM happy so that they'd buy our languages. > > The UNIX contracts were all done when Bell Labs was regulated and couldn't sell Unix into the commerical marketplace. So although they wrote it and were paid royalties, they couldn't develop it in competition to us. But after a few years that changed. Bell was degregulated and now they were selling Unix directly, in competition to us! They might sell it for cheaper than we had to pay them in royalties! But that wasn't the real killer, the real killer was the Bell now controlled the standard. If we wrote an API extension that did X, and Bell wrote an incompatible one that did Y, which one would people write for? The ISVs know that AT&T was a very big company and that they'd written the original, so they'd believe that AT&T controlled the standard, not MS, and that belief would then define reality. So we'd always just be waiting for what AT&T announced and then frantically trying to duplicate it. > > Bill Gates knew, right away, that there was no strong future in Unix for us any more. Fortunately at that time, DOS was taking off and we were learning, along with everyone else, about the power of standards. So the primary OS team - the Unix guys - joined with the secondary OS team - the DOS guys - and the earliest versions of OS/2 were born. (This was before IBM came on board, so it wasn't called OS/2!) > Marc Rochkind > Regarding the Microsoft/UNIX connection, while AT&T was central in the UNIX world, Microsoft is famous for their volume, I find myself wondering if Microsoft ever considered working *with* AT&T as an angle. Would this have run afoul of their relationship with IBM? I understand it that AT&T was trying to posture themselves as an IBM competitor in the hardware market in the ATTIS era, so I could see this factoring into Microsoft pulling out rather than espousing an angle of "If you can't beat them, join them." Again though, given their volume, I could see an alternate timeline where Microsoft approached AT&T and AT&T was more than willing to leverage a relationship with Microsoft given the uptake of Xenix. AT&T would eventually plunder Xenix for bits leading up to SVR4 anyway, granted this was many years later with more perspective. Another angle I've pondered on too is if Microsoft would've been amenable to that sort of thing but AT&T wouldn't have. They had just settled a huge anti-trust case. Pairing themselves with the single largest distributor of UNIX may have been to scarily close to cornering a market for their comfort, so maybe even if Microsoft had considered that, I could see trepidation on AT&Ts part regarding high-profile integration with an operation like Microsoft at the time... Cool stuff though, I've been studying this point of history a bit lately WRT the UTS/386 connection brought up recently. In a similar "don't mess with IBM" vein, it's had me wondering if Intel management would've been sketchy about using UTS for anything since Amdahl was a prominent IBM competitor. I get the impression that industry players that managed to curry IBMs favor somehow then had to tiptoe carefully around anything that might smell of engaging with competition. Just my view in hindsight though, as always, I wasn't there, I'm just fascinated with the conditions that lead to the world I live in :) - Matt G. From johnl at taugh.com Sat Dec 7 04:33:11 2024 From: johnl at taugh.com (John Levine) Date: 6 Dec 2024 18:33:11 -0000 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: Message-ID: According to Al Kossow : >On 12/6/24 8:38 AM, Arthur Krewat wrote: >> /his was before IBM came on board/ > >and IBM had no interest in the 386 at the time, which set the >wheels in motion for Gates to acquire Cutler and DECwest That was oddly shortsighted of IBM. Was it 16 bits is enough for anything you'd do on your desktop, or 32 bits is too close to competing with our big machines? -- Regards, John Levine, johnl at taugh.com, Primary Perpetrator of "The Internet for Dummies", Please consider the environment before reading this e-mail. https://jl.ly From henry.r.bent at gmail.com Sat Dec 7 05:29:53 2024 From: henry.r.bent at gmail.com (Henry Bent) Date: Fri, 6 Dec 2024 14:29:53 -0500 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: Message-ID: On Fri, 6 Dec 2024 at 12:13, Al Kossow wrote: > On 12/6/24 8:38 AM, Arthur Krewat wrote: > > /his was before IBM came on board/ > > and IBM had no interest in the 386 at the time > Are you referring strictly to the 386 as a UNIX machine? The 386 shipped in 1986 and the IBM PS/2 Model 80 (with a 386) shipped in '87. My assumption is that the driving factor in keeping the lower end PS/2 machines around was cost - at launch, a PS/2 with an 8086 was $2300 while one with a 386 was over $10k. Of course, the 386 PS/2 was initially targeted at OS/2 and not UNIX; from what I can find AIX PS/2 didn't ship until at least late 1988. -Henry -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Sat Dec 7 07:46:02 2024 From: tuhs at tuhs.org (Chet Ramey via TUHS) Date: Fri, 6 Dec 2024 13:46:02 -0800 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: <87ttbhzlho.fsf@gmail.com> References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <2fd6a361-b26f-4ca4-9c65-b6ba0559644e@case.edu> <202412052055.4B5KtYf1781856@freefriends.org> <002cf6d3-0ed2-46f2-9fae-19a3e10d7b9c@case.edu> <87ttbhzlho.fsf@gmail.com> Message-ID: <0c3c3028-9293-4251-9e68-4874654c561b@case.edu> On 12/5/24 7:46 PM, Alexis wrote: > Can anyone point me to any discussions about process substitution > potentially becoming part of POSIX? It's never come up in any serious way, maybe as a wish list item for some people on the mailing list. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature.asc Type: application/pgp-signature Size: 203 bytes Desc: OpenPGP digital signature URL: From heinz at osta.com Sat Dec 7 08:04:26 2024 From: heinz at osta.com (Heinz Lycklama) Date: Fri, 6 Dec 2024 14:04:26 -0800 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: <26vRiRS2sO-BAumb5V9ClzUqNJTMHE6ISl04n95AGMV2z3Kx_mmG9qOIIsLMGAAWJKprznaC9rGeI7_bX842X__q39Ind8b5O0JfuSkoLkk=@protonmail.com> References: <26vRiRS2sO-BAumb5V9ClzUqNJTMHE6ISl04n95AGMV2z3Kx_mmG9qOIIsLMGAAWJKprznaC9rGeI7_bX842X__q39Ind8b5O0JfuSkoLkk=@protonmail.com> Message-ID: To add a footnote to this discussion regarding MS and UNIX. We started work on the "UNIX standard" in 1981 under the auspices of /usr/group. Many of the small UNIX system vendors and application developers joined the effort right up front, including UNIX-like vendors. It took a little longer to get AT&T to participate, and then even longer to get IBM to join. MS participated in our efforts because of their Xenix products for a while but then suddenly pulled out. Don't remember the exact date, but they never joined us again. The Proposed /usr/group Standard published in January 1984 includes one name from MS in its Working Group membership. The final /usr/group Standard was published in November 1984 still includes the one name from MS among its 60+ members. Letwin's 1995 post explains why MS withdrew from the /usr/group Standards effort, which is the foundation of today's POSIX Standard published by the IEEE. Heinz On 12/6/2024 9:28 AM, segaloco via TUHS wrote: > On Friday, December 6th, 2024 at 8:10 AM, Marc Rochkind wrote: > >> I just came across a 1995 post from Gordon Letwin, early Microsoft employee and lead architect of OS/2, about the history of OS/2. There are a few paragraphs in it about Microsoft and UNIX. Here's Letwin's post: >> >> https://gunkies.org/wiki/Gordon_Letwin_OS/2_usenet_post >> >> And the UNIX-related paragraphs: >> >> It's extremely hard to do development work on an operating system when someone else controls the standard. "Control" in this case is a matter of public perception. For example, Microsoft was once very big in the Unix world. In fact, we considered it our candidate for the future desktop operating system, when machines got powerful enough to run something good. We were the worlds biggest seller of Unix systems. DOS was, when we first wrote it, a one-time throw-away product intended to keep IBM happy so that they'd buy our languages. >> >> The UNIX contracts were all done when Bell Labs was regulated and couldn't sell Unix into the commerical marketplace. So although they wrote it and were paid royalties, they couldn't develop it in competition to us. But after a few years that changed. Bell was degregulated and now they were selling Unix directly, in competition to us! They might sell it for cheaper than we had to pay them in royalties! But that wasn't the real killer, the real killer was the Bell now controlled the standard. If we wrote an API extension that did X, and Bell wrote an incompatible one that did Y, which one would people write for? The ISVs know that AT&T was a very big company and that they'd written the original, so they'd believe that AT&T controlled the standard, not MS, and that belief would then define reality. So we'd always just be waiting for what AT&T announced and then frantically trying to duplicate it. >> >> Bill Gates knew, right away, that there was no strong future in Unix for us any more. Fortunately at that time, DOS was taking off and we were learning, along with everyone else, about the power of standards. So the primary OS team - the Unix guys - joined with the secondary OS team - the DOS guys - and the earliest versions of OS/2 were born. (This was before IBM came on board, so it wasn't called OS/2!) >> Marc Rochkind >> > Regarding the Microsoft/UNIX connection, while AT&T was central in the UNIX world, Microsoft is famous for their volume, I find myself wondering if Microsoft ever considered working *with* AT&T as an angle. Would this have run afoul of their relationship with IBM? I understand it that AT&T was trying to posture themselves as an IBM competitor in the hardware market in the ATTIS era, so I could see this factoring into Microsoft pulling out rather than espousing an angle of "If you can't beat them, join them." Again though, given their volume, I could see an alternate timeline where Microsoft approached AT&T and AT&T was more than willing to leverage a relationship with Microsoft given the uptake of Xenix. AT&T would eventually plunder Xenix for bits leading up to SVR4 anyway, granted this was many years later with more perspective. > > Another angle I've pondered on too is if Microsoft would've been amenable to that sort of thing but AT&T wouldn't have. They had just settled a huge anti-trust case. Pairing themselves with the single largest distributor of UNIX may have been to scarily close to cornering a market for their comfort, so maybe even if Microsoft had considered that, I could see trepidation on AT&Ts part regarding high-profile integration with an operation like Microsoft at the time... > > Cool stuff though, I've been studying this point of history a bit lately WRT the UTS/386 connection brought up recently. In a similar "don't mess with IBM" vein, it's had me wondering if Intel management would've been sketchy about using UTS for anything since Amdahl was a prominent IBM competitor. I get the impression that industry players that managed to curry IBMs favor somehow then had to tiptoe carefully around anything that might smell of engaging with competition. Just my view in hindsight though, as always, I wasn't there, I'm just fascinated with the conditions that lead to the world I live in :) > > - Matt G. From steve at quintile.net Sat Dec 7 08:07:00 2024 From: steve at quintile.net (Steve Simon) Date: Fri, 6 Dec 2024 22:07:00 +0000 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact Message-ID: <034FC1E9-1D6C-42B9-B1B1-8AA03379E70E@quintile.net> helios supported DAG shell syntax. helios was a unix style os (was it actually posix compliant?) for networks of transputers. -Steve From ylee at columbia.edu Sat Dec 7 08:43:09 2024 From: ylee at columbia.edu (Yeechang Lee) Date: Fri, 6 Dec 2024 14:43:09 -0800 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: Message-ID: <26451.32253.73591.816292@dobie-old.ylee.org> John Levine says: > That was oddly shortsighted of IBM. Was it 16 bits is enough for > anything you'd do on your desktop, or 32 bits is too close to > competing with our big machines? Compaq got its Deskpro 386 out by late 1986. IBM didn't see the urgency and released the PS/2 Model 80 in June 1987. Not just IBM; HP, for example, in 1987 was still saying publicly that it was evaluating when and how to release its own 386 system. Compaq's move panicked smaller competitors who didn't need to preserve their dignity and knew what the computer meant, with many showing hastily built prototypes at November 1986 Comdex. While Microsoft did help Compaq while designing Deskpro 386, and Gates attended the computer's announcement, I don't think it affected its plans for Xenix and OS/2. The announcement did establish Compaq as arguably the standard setter in IBM's place by 1990, or more accurately proved that IBM was no longer the standard setter. Had Dell been the first out with a 386 box that might have affected its plans for Dell Unix, but Compaq never had its own operating system until the DEC acquisition. From tuhs at tuhs.org Sat Dec 7 08:58:00 2024 From: tuhs at tuhs.org (Warren Toomey via TUHS) Date: Sat, 7 Dec 2024 08:58:00 +1000 Subject: [TUHS] A re-analysis of some DECtapes from dmr Message-ID: All, I received this detailed e-mail from Yufeng Gao who has done a great job at analysing some of the DECtapes that Dennis made available to the Unix Archive. I'm sure many of you would be interested and, possibly, could provide some feedback on his work. A tarball of his efforts are here: https://mega.nz/file/1cJFTJxT#BgszYuFc_ebSMaQXAIG5b2NtaGInPWMoaxExPM5Lr9Y Cheers, Warren P.S. I have a follow-up e-mail coming ... ----- Forwarded message from Yufeng Gao ----- Subject: UNIX DECtapes from Ritchie Hi Warren, I am writing you this email after seeing the DECtapes from Dennis Ritchie that you put up last year. I found them while playing around with the early C compilers on Ritchie's site. After some research into the tapes, I have written a parser to parse them and convert them into tarballs suitable for preservation. Most importantly, the 'dmr' tape and 's1' tape are currently not decoded properly. The biggest issue with the current decoding of the 's1' tape is, well, there is none. After quickly glancing over the tape, I have concluded that it is in fact not one of the middle tapes of an rkd(1) backup, but a copy of UNIX INIT DECtape from a version of UNIX between V1 and V2. I have extracted its contents: Mode UID GID Size Date Name ---------- --- --- ----- ------------------- ------------ -rw-rw-rw- 0 0 512 1970-01-01 00:00:00 [vcboot] -rw-rw-rw- 0 0 2048 1970-01-01 00:00:00 [bos] -rw-rw-rw- 0 0 12288 1970-01-01 00:00:00 [wunix] -rw-rw-rw- 0 0 12288 1970-01-01 00:00:00 [cunix] -rw-rw-rw- 0 0 3072 1970-01-01 00:00:00 [unassigned] -rwxr-xr-x 1 0 424 1970-01-01 00:00:00 /etc/init -rwxrwxrwx 3 0 446 1970-01-01 00:00:00 /etc/getty -rwxr-xr-x 3 0 82 1970-01-01 00:00:00 /bin/chmod -rwsr-xr-x 0 0 794 1970-01-01 00:00:00 /bin/date -rwsr-xr-x 0 0 1290 1970-01-01 00:00:00 /bin/login -rwsr-xr-x 0 0 232 1970-01-01 00:00:00 /bin/mkdir -rwxr-xr-x 1 0 954 1970-01-01 00:00:00 /bin/sh -rwsr-xr-x 0 0 3678 1970-01-01 00:00:00 /bin/tap -rwxr-xr-x 3 0 2010 1970-01-01 00:00:00 /bin/ls 578 blocks, 98 (16.96%) used, 480 (83.04%) free. B = boot; D = directroy; . = free; X = file data; O = bos; W = wunix; C = cunix; S = rf slack; U = unassigned program; |0123456789ABCDEF --+---------------- 00|BOOOOWWWWWWWWWWW 01|WWWWWWWWWWWWWCCC 02|CCCCCCCCCCCCCCCC 03|CCCCCUUUUUUSSSSS 04|SDXDXDXDXXDXXXDX 05|DXXDXXXXXXXXDXXX 06|XD.............. 07|................ <...> What is significant here is it contains the vcboot and bos programs mentioned in bproc(7), as well as a Warm and a Cold UNIX kernel. This version of UNIX appears to be between V1 and V2, which I believe is the earliest to exist in binary form. The reason why I say it's between V1 and V2 is its bos program accepts the following console switches compared to V1 and V2: | UNIX V1 | s1 | UNIX V2 ----------------------+----------+-------+--------- Warm boot | [1]73700 | ??? | ??? Cold boot | 1 | 1 | 1 Unassigned 3K prog | 2 | 2 | Dump core & halt | 10 | 10 | 10 Boot from RK | | 20 | 20 Dump core & warm boot | | 40 | 40 Boot from paper tape | 0 | 0 | 0 Load DEC loaders | 57500 | 57500 | 77500 ----------------------+----------+-------+--------- UNIX load address | 400 | 400 | 600 ----------------------+----------+-------+--------- The boot-from-unassigned-3K-program option was probably in the process of being depreciated, as the 3K of space is also loaded as a part of the Cold UNIX despite it not being used by the Cold UNIX kernel. The list of files on the 's1' tape also appear to be between V1 and V2: | V1 | s1 | V2 -----------+-----+-----+----- /etc/init | Yes | Yes | Yes /etc/getty | No | Yes | Y/N /bin/chmod | Yes | Yes | Yes /bin/chown | Yes | No | No /bin/date | No | Yes | Yes /bin/login | No | Yes | Yes /bin/cp | Yes | No | No /bin/ln | Yes | No | No /bin/ls | Yes | Yes | Yes /bin/mkdir | Yes | Yes | Yes /bin/mv | Yes | No | No /bin/rm | Yes | No | No /bin/rmdir | Yes | No | No /etc/mount | No | No | Yes /bin/sh | Yes | Yes | Yes /bin/stat | Yes | No | No /bin/tap | Yes | Yes | Yes -----------+-----+-----+----- Given that the files on this tape are identical to the same files on the 's2' tape and that the files on the 's2' tape are also sort of between V1 and V2, and that this tape is named 's1' while the other is named 's2', they are probably from the same version of UNIX. I have tried booting the 's1' tape using SIMH, but unfortunately it did not work and I have not yet attempted to debug it. Next I'm going to talk about the 'dmr' tape. It’s interesting, most of it was written by tap(1) running under UNIX V1-V2, but files in the ./paper directory were written by tap(1) running under UNIX V4. When decoded using the tap(1) program itself, the timestamps and access modes of those files are garbled. My program corrected the timestamps and access modes in the converted tar(1) archive. The same goes for the 'nsys' tape. The tar(1) conversion of 's2' (/Archive/Distributions/Research/1972_stuff/s2-bits.tar.gz) in the TUHS archives is missing some files in the /etc directory - compare it with mine and Ritchie's (Ritchie's has wrong timestamps and access modes, however). Speaking of timestamps and the 's2' tape, the timestamps outputted by tap(1) are inaccurate because 1972 is a leap year and the ctime(3) function hard coded 28 for the number of days in February. I have attached an archive of my tar(1) conversion of each tape, as well as tarballs containing all the slack space data and unused blocks which I dumped for data recovery and forensic analysis purposes. There are some interesting bits and pieces in the unused blocks of various tapes, but I have not had time to look into them in detail. Just a very quick summary: * The 'config' tape has leftover of a dictionary file. * The 'dmr' tape has some assembler manual ?roff leftover. * The 'dmr2' tape has some paper ?roff leftover. * The 'e-pi' tape has a lot of interesting binary stuff, including bits and pieces of a B compiler from what I can tell. * The 'ken' tape has some C source fragments and Thompson’s mail templates and some UNIX manual ?roff leftover. * The 'ken-du' tape has some paper ?roff leftover. * The 'ken-sky' tape has some binary leftover. * The 'nsys' tape has some source code leftover of what seems to be the kernel. * The 's1' tape has some source code leftover of what seems to be userland programs. * The 'scratch' tape has some binary and source listing and ?roff leftover. * The 'unix' tape has some binary leftover. I hope that these newly extracted stuff could be put up in the TUHS archives :). If the attachment in this email somehow doesn't come through, I have also uploaded the archive [1]here. I have a disassembly of vcboot and bos from 's1', they may help with the UNIX V1 reconstruction project - let me know if you need them. If you can get the 's1' kernel to boot, I'd like to also receive an update. Sincerely, Yufeng Gao ----- End forwarded message ----- From tuhs at tuhs.org Sat Dec 7 09:01:48 2024 From: tuhs at tuhs.org (Warren Toomey via TUHS) Date: Sat, 7 Dec 2024 09:01:48 +1000 Subject: [TUHS] A re-analysis of some DECtapes from dmr In-Reply-To: References: Message-ID: On 7/12/24 08:58, Warren Toomey via TUHS wrote: > All, I received this detailed e-mail from Yufeng Gao who has done a great > job at analysing some of the DECtapes that Dennis made available to the > Unix Archive.P.S. I have a follow-up e-mail coming ... Here is a follow-up e-mail from Yufeng, I just needed to ensure the colours got preserved :-) Cheers, Warren ---- From Yufeng Gao ---- If you're going to share my email, please consider adding the following information about the s1 and s2 files being from the same version of UNIX: "The maki.s source and a.out executable found in /usr/sys is the exact same maki program used to create the INIT tape, judging by the unzeroed/leftover bytes at the end of vcboot." And the row highlighted in green from the following table to the one from my last email:                | UNIX V1  | s1    | UNIX V2 ----------------------+----------+-------+--------- Warm boot             | [1]73700 | ???   | ??? Cold boot             | 1        | 1     | 1 Unassigned 3K prog    | 2        | 2     | Dump core & halt      | 10       | 10    | 10 Boot from RK          |          | 20    | 20 Dump core & warm boot |          | 40    | 40 Boot from paper tape  | 0        | 0     | 0 Load DEC loaders      | 57500    | 57500 | 77500 ----------------------+----------+-------+--------- *BOS load address      | 54000    | 54000 | 154000* UNIX load address     | 400      | 400   | 600 ----------------------+----------+-------+--------- Sorry if I sound like I'm asking for too much, the history is quite important so I want everything to be in the best possible state :). Sincerely, Yufeng Gao -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at oneiros.de Sat Dec 7 10:53:08 2024 From: martin at oneiros.de (=?UTF-8?Q?Martin_Schr=C3=B6der?=) Date: Sat, 7 Dec 2024 01:53:08 +0100 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact In-Reply-To: <034FC1E9-1D6C-42B9-B1B1-8AA03379E70E@quintile.net> References: <034FC1E9-1D6C-42B9-B1B1-8AA03379E70E@quintile.net> Message-ID: Am Fr., 6. Dez. 2024 um 23:07 Uhr schrieb Steve Simon : > helios supported DAG shell syntax. Wikipedia: "What is not immediately apparent is that Helios extends the notion of Unix pipes into a language called Component Distribution Language (CDL). In CDL, a typical Unix shell pipeline such as more is called a task force, and is transparently distributed by the Task Force Manager server across the available CPUs. CDL extends traditional Unix syntax with additional operators for bi-directional pipes, sequential and parallel process farm operators, load balancing and resource management." > helios was a unix style os (was it actually posix compliant?) for networks of transputers. Mostly. "The Unix compatibility library described in chapter 5, Compatibility, implements functions which are largely compatible with the Posix standard interfaces. The library does not include the entire range of functions provided by the Posix standard, because some standard functions require memory management or, for various reasons, cannot be implemented on a multi-processor system. The reader is therefore referred to IEEE Std 1003.1-1988, IEEE Standard Portable Operating System Interface for Computer Environments, which is available from the IEEE Service Center, 445 Hoes Lane, P.O. Box 1331, Piscataway, NJ 08855-1331, USA. It can also be obtained by telephoning USA (201) 9811393" http://www.transputer.net/hbooks/techref/hbk.pdf Does that number still connect to the IEEE? :-) Best Martin From henry.r.bent at gmail.com Sat Dec 7 13:11:26 2024 From: henry.r.bent at gmail.com (Henry Bent) Date: Fri, 6 Dec 2024 22:11:26 -0500 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: <26451.32253.73591.816292@dobie-old.ylee.org> References: <26451.32253.73591.816292@dobie-old.ylee.org> Message-ID: On Fri, 6 Dec 2024 at 17:53, Yeechang Lee wrote: > John Levine says: > > That was oddly shortsighted of IBM. Was it 16 bits is enough for > > anything you'd do on your desktop, or 32 bits is too close to > > competing with our big machines? > > Compaq got its Deskpro 386 out by late 1986. IBM didn't see the urgency > and released the PS/2 Model 80 in June 1987. Not just IBM; HP, for example, > in 1987 was still saying publicly that it was evaluating when and how to > release its own 386 system. > > Compaq's move panicked smaller competitors who didn't need to preserve > their dignity and knew what the computer meant, with many showing hastily > built prototypes at November 1986 Comdex. > > While Microsoft did help Compaq while designing Deskpro 386, and Gates > attended the computer's announcement, I don't think it affected its plans > for Xenix and OS/2. The announcement did establish Compaq as arguably the > standard setter in IBM's place by 1990, or more accurately proved that IBM > was no longer the standard setter. Had Dell been the first out with a 386 > box that might have affected its plans for Dell Unix, but Compaq never had > its own operating system until the DEC acquisition. > I may be showing my ignorance here, but Compaq rushed to market a 386 machine so it could run... what? 16 bit DOS? Other 16 bit operating systems? It's kind of astonishing to me that no one had a 32 bit operating system ready for the 386 PC market, especially given that Intel had released the chip to developers a year earlier. SVR2 was readily available as a porting base but it appears that pretty much everyone dropped the ball on having a UNIX ready for a fairly powerful $10k machine with a clearly established market acceptance (from any vendor, not just Compaq or IBM). -Henry -------------- next part -------------- An HTML attachment was scrubbed... URL: From ylee at columbia.edu Sat Dec 7 13:38:16 2024 From: ylee at columbia.edu (Yeechang Lee) Date: Fri, 6 Dec 2024 19:38:16 -0800 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: <26451.32253.73591.816292@dobie-old.ylee.org> Message-ID: <26451.49960.376499.18073@dobie-old.ylee.org> Henry Bent says: > I may be showing my ignorance here, but Compaq rushed to market a > 386 machine so it could run... what?  16 bit DOS?  Other 16 bit > operating systems? Yes. Contemporary news articles on the announcement discuss this, stating that a) no current software takes full advantage of the 386's power, and b) regardless, those who need the added horsepower would still benefit. The Deskpro 386's strong sales proved that this view was correct. > It's kind of astonishing to me that no one had a 32 bit operating > system ready for the 386 PC market IBM, as mentioned, did not think the market wanted or needed a 386-based computer. In 1986 its fastest PC was the 286-based IBM AT, and in the two years since no one had released any software requiring the 286; everyone treated it as a faster 8088 or 8086. The AT was in 1984 an aberration, really the only time in the IBM PC's history that the IBM product was state of the art. IBM otherwise followed a PC product lifecycle similar to that of its big iron; the original 1981 IBM PC was still sold until the PS/2 introduction in 1987, for example. The company was very surprised when its 1985 discontinuation of the PCjr upset customers, because IBM expected that its normal practice of promising ongoing support for a period of years into the future would be sufficient. The only vendors with credibility to establish a new industry standard with an operating system in 1984 or 1986 were IBM and Microsoft. Had AT&T's entry into the computer market after divestiture not occurred, Microsoft would likely have continued pushing Xenix and might well have had a 386-specific version ready soon after the Deskpro 386. Such a product would in time surely have used the virtual 8086 hardware feature to execute DOS software on Xenix, akin to contemporary Windows's similar feature but with the advantages of a) actually working and b) with preemptive multitasking. From mrochkind at gmail.com Sat Dec 7 13:55:36 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Fri, 6 Dec 2024 20:55:36 -0700 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: <26451.32253.73591.816292@dobie-old.ylee.org> Message-ID: I'm sure Compaq was thinking ahead. They would be very aware of Microsoft's Windows plans. Buyers also think ahead, especially corporate buyers, who are the real customers. Marc On Fri, Dec 6, 2024, 8:11 PM Henry Bent wrote: > On Fri, 6 Dec 2024 at 17:53, Yeechang Lee wrote: > >> John Levine says: >> > That was oddly shortsighted of IBM. Was it 16 bits is enough for >> > anything you'd do on your desktop, or 32 bits is too close to >> > competing with our big machines? >> >> Compaq got its Deskpro 386 out by late 1986. IBM didn't see the urgency >> and released the PS/2 Model 80 in June 1987. Not just IBM; HP, for example, >> in 1987 was still saying publicly that it was evaluating when and how to >> release its own 386 system. >> >> Compaq's move panicked smaller competitors who didn't need to preserve >> their dignity and knew what the computer meant, with many showing hastily >> built prototypes at November 1986 Comdex. >> >> While Microsoft did help Compaq while designing Deskpro 386, and Gates >> attended the computer's announcement, I don't think it affected its plans >> for Xenix and OS/2. The announcement did establish Compaq as arguably the >> standard setter in IBM's place by 1990, or more accurately proved that IBM >> was no longer the standard setter. Had Dell been the first out with a 386 >> box that might have affected its plans for Dell Unix, but Compaq never had >> its own operating system until the DEC acquisition. >> > > I may be showing my ignorance here, but Compaq rushed to market a 386 > machine so it could run... what? 16 bit DOS? Other 16 bit operating > systems? It's kind of astonishing to me that no one had a 32 bit operating > system ready for the 386 PC market, especially given that Intel had > released the chip to developers a year earlier. SVR2 was readily available > as a porting base but it appears that pretty much everyone dropped the ball > on having a UNIX ready for a fairly powerful $10k machine with a clearly > established market acceptance (from any vendor, not just Compaq or IBM). > > -Henry > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sjenkin at canb.auug.org.au Sat Dec 7 16:07:54 2024 From: sjenkin at canb.auug.org.au (sjenkin at canb.auug.org.au) Date: Sat, 7 Dec 2024 17:07:54 +1100 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact In-Reply-To: <034FC1E9-1D6C-42B9-B1B1-8AA03379E70E@quintile.net> References: <034FC1E9-1D6C-42B9-B1B1-8AA03379E70E@quintile.net> Message-ID: <5D2A3CE6-96C7-4A03-A0A7-7F645BAE637D@canb.auug.org.au> > On 7 Dec 2024, at 09:07, Steve Simon wrote: > > > helios supported DAG shell syntax. > > helios was a unix style os (was it actually posix compliant?) for networks of transputers. > > -Steve For anyone playing along at home, some links I turned up this afternoon. There are other O/S of the same name, using the capitalisation which the Perihelion version eschewed. One for Arduino and another Sel4 related microkernel Wikipedia - already mentioned [ with incorrect capitalisation? ] Helios books: many texts, PDF & html, available + names of many resources Helios Shell CDL - A Distributed Language for Helios CDL was mentioned by others, source code might still be available in Helios-NG Repo Helios-NG [ untouched for years ] This is a project for breathing new live in Helios, an OS from the 90's. Helios was developed by the (now defunct) company called Perihelion Ltd., mainly targeting the INMOS Transputer but later adding other CPUs like the ARM series or TMS320c4x DSPs when INMOS' decline became clear. Legal? I got the permission from Perhelions fromer CEO Tim King and main programmer Nick Garnett to "[...] do whatever you want with the Helios sources." So here we are: "Helios Next Generation” or Helios-NG for short, put under GPL v3. Author's Helios-NG page, 2014. ’The Plan' Transputer board, 2024 To make a long story short: This is basically a design, where the ATARI Mega-ST is used as a boot device and after that just handles file- and user-I/O. The Transputer is attached to the ST via DMA and runs the Helios OS and has direct access to the graphics controller called ‘Blossom’. Totally different concept. Author ‘about' ABOUT ME Hi! I’ll make this short… or at least try to. My name is Axel Muhr. Born 1970 I’m an fairly old geek compared to todays Web 2.0 standards. Let’s say I do remember how to cope with 4k RAM and considered a 174kb 5.25″ Floppy quite a big storage device… not mentioning it’s insane speed. -- Steve Jenkin, IT Systems and Design 0412 786 915 (+61 412 786 915) PO Box 38, Kippax ACT 2615, AUSTRALIA mailto:sjenkin at canb.auug.org.au http://members.tip.net.au/~sjenkin From henry.r.bent at gmail.com Sat Dec 7 18:46:49 2024 From: henry.r.bent at gmail.com (Henry Bent) Date: Sat, 7 Dec 2024 03:46:49 -0500 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact In-Reply-To: <5D2A3CE6-96C7-4A03-A0A7-7F645BAE637D@canb.auug.org.au> References: <034FC1E9-1D6C-42B9-B1B1-8AA03379E70E@quintile.net> <5D2A3CE6-96C7-4A03-A0A7-7F645BAE637D@canb.auug.org.au> Message-ID: On Sat, 7 Dec 2024 at 01:08, wrote: > > > On 7 Dec 2024, at 09:07, Steve Simon wrote: > > Wikipedia - already mentioned [ with incorrect capitalisation? ] > > Indeed. While the article uses correct capitalization throughout, the title of the article is incorrect. I have requested a move to the correct capitalization, which unfortunately (but probably understandably) in Wikipedia land is an involved process. -Henry -------------- next part -------------- An HTML attachment was scrubbed... URL: From lars at nocrew.org Sat Dec 7 19:32:58 2024 From: lars at nocrew.org (Lars Brinkhoff) Date: Sat, 07 Dec 2024 09:32:58 +0000 Subject: [TUHS] A re-analysis of some DECtapes from dmr In-Reply-To: (Warren Toomey via TUHS's message of "Sat, 7 Dec 2024 09:01:48 +1000") References: Message-ID: <7wr06jsur9.fsf@junk.nocrew.org> Is the kernel for a PDP-11/20 or /45? If /20, does it use the KS11 for memory mapping? From davida at pobox.com Sat Dec 7 19:56:12 2024 From: davida at pobox.com (David Arnold) Date: Sat, 7 Dec 2024 20:56:12 +1100 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact In-Reply-To: References: Message-ID: <9D33242B-DA95-4927-A73E-CC22CDD5723A@pobox.com> > On 7 Dec 2024, at 11:53, Martin Schröder wrote: > > Am Fr., 6. Dez. 2024 um 23:07 Uhr schrieb Steve Simon : >> helios supported DAG shell syntax. > > Wikipedia: > "What is not immediately apparent is that Helios extends the notion of > Unix pipes into a language called Component Distribution Language > (CDL). I have somewhere the CDL reference: a small volume, perhaps 50 pages. If I recall correctly, there’s some discussion in the two Helios books as well. I can dig them up if there’s interest. <…> >> helios was a unix style os (was it actually posix compliant?) for networks of transputers. > > Mostly. I think it was missing fork(), although I believe it had a vfork(). It was able to run a lot of the free software of the time with a little effort. There was a GCC port, gnumake, MicroEmacs, X11R4 (? Maybe R5?) It was an interesting OS. Not unlike Plan9: its file server protocol is roughly analogous to 9p. d From jsg at jsg.id.au Sat Dec 7 20:39:52 2024 From: jsg at jsg.id.au (Jonathan Gray) Date: Sat, 7 Dec 2024 21:39:52 +1100 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: <26451.32253.73591.816292@dobie-old.ylee.org> Message-ID: On Fri, Dec 06, 2024 at 10:11:26PM -0500, Henry Bent wrote: > On Fri, 6 Dec 2024 at 17:53, Yeechang Lee wrote: > > > John Levine says: > > > That was oddly shortsighted of IBM. Was it 16 bits is enough for > > > anything you'd do on your desktop, or 32 bits is too close to > > > competing with our big machines? > > > > Compaq got its Deskpro 386 out by late 1986. IBM didn't see the urgency > > and released the PS/2 Model 80 in June 1987. Not just IBM; HP, for example, > > in 1987 was still saying publicly that it was evaluating when and how to > > release its own 386 system. > > > > Compaq's move panicked smaller competitors who didn't need to preserve > > their dignity and knew what the computer meant, with many showing hastily > > built prototypes at November 1986 Comdex. > > > > While Microsoft did help Compaq while designing Deskpro 386, and Gates > > attended the computer's announcement, I don't think it affected its plans > > for Xenix and OS/2. The announcement did establish Compaq as arguably the > > standard setter in IBM's place by 1990, or more accurately proved that IBM > > was no longer the standard setter. Had Dell been the first out with a 386 > > box that might have affected its plans for Dell Unix, but Compaq never had > > its own operating system until the DEC acquisition. > > > > I may be showing my ignorance here, but Compaq rushed to market a 386 > machine so it could run... what? 16 bit DOS? Other 16 bit operating > systems? It's kind of astonishing to me that no one had a 32 bit operating > system ready for the 386 PC market, especially given that Intel had > released the chip to developers a year earlier. SVR2 was readily available > as a porting base but it appears that pretty much everyone dropped the ball > on having a UNIX ready for a fairly powerful $10k machine with a clearly > established market acceptance (from any vendor, not just Compaq or IBM). The 1986 press release for the Deskpro 386 mentioned 386 Xenix was planned for 1987. https://www.latimes.com/archives/la-xpm-1986-09-10-fi-13177-story.html Intel and AT&T had ISC do a 386 port for SVR3. "The 386/ix is based on Release 3.0, which was developed by Interactive Systems Corp. Santa Monica, Calif., under contract to Intel and AT&T. The code was tested through an extensive beta program managed by Intel (with more than 60 80386 beta sites)." Mini-Micro Systems, January 1988, p 45 https://archive.org/details/bitsavers_MiniMicroS_59292072/page/44/mode/2up https://bitsavers.org/magazines/Mini-Micro_Systems/198801.pdf AT&T sold rebadged Olivetti machines with SVR3 in 1987: "AT&T 6386 WGS is today's only 80386-based system to take full advantage of its 32-bit architecture" https://bitsavers.org/pdf/att/6386_wgs/6386_WGS_Brochure.pdf ISC work was also used by Microport: "Microport Runtime System V/386 is based on a version of Unix for the 80386 carried out by Interactive Technologies for AT&T and Intel." Microport to Ship Version of Unix for 386 InfoWorld, Volume 9, Issue 9, 2 Mar 1987, p 3 https://books.google.com/books?id=1TAEAAAAMBAJ&pg=PA3 From henry.r.bent at gmail.com Sat Dec 7 22:10:48 2024 From: henry.r.bent at gmail.com (Henry Bent) Date: Sat, 7 Dec 2024 07:10:48 -0500 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact In-Reply-To: <9D33242B-DA95-4927-A73E-CC22CDD5723A@pobox.com> References: <9D33242B-DA95-4927-A73E-CC22CDD5723A@pobox.com> Message-ID: On Sat, 7 Dec 2024 at 04:56, David Arnold wrote: > > It was able to run a lot of the free software of the time with a little > effort. There was a GCC port, gnumake, MicroEmacs, X11R4 (? Maybe R5?) > I was curious about the GCC port as I've spent far, far too much time looking through GCC support for various platforms and I have no memory of this target. Turns out it was an entirely independent effort, and probably a one-off: https://groups.google.com/g/comp.sys.transputer/c/Zt6WNxneS08/m/ZBRytllF_X8J . The standard C compiler was a different beast, and I was not immediately able to recognize its lineage; it may have been developed from scratch. A mostly complete archive of Helios sources is here: https://github.com/axelmuhr/Helios-NG -Henry -Henry -------------- next part -------------- An HTML attachment was scrubbed... URL: From henry.r.bent at gmail.com Sat Dec 7 22:34:07 2024 From: henry.r.bent at gmail.com (Henry Bent) Date: Sat, 7 Dec 2024 07:34:07 -0500 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: <26451.32253.73591.816292@dobie-old.ylee.org> Message-ID: This is all very enlightening, thank you. Comments inline... On Sat, 7 Dec 2024 at 05:39, Jonathan Gray wrote: > The 1986 press release for the Deskpro 386 mentioned 386 Xenix > was planned for 1987. > https://www.latimes.com/archives/la-xpm-1986-09-10-fi-13177-story.html I found https://www.tech-insider.org/unix/research/1987/0902.html which includes the following quote: "UNIX System V and the 80386 are a perfect technological match," said Bill Gates, chairman of Microsoft Corporation, in remarks at AT&T's press conference here. AT&T and Microsoft are developing a new version of UNIX System V for the 80386 chip that will run XENIX System V as well as UNIX System V applications. This is September 1987, so perhaps Microsoft's abandonment of Xenix was not as early as I had thought. Though this does imply that the Xenix port was not ready at that point, and perhaps was ultimately abandoned by Microsoft. > Intel and AT&T had ISC do a 386 port for SVR3. > > "The 386/ix is based on Release 3.0, which was developed by Interactive > Systems Corp. Santa Monica, Calif., under contract to Intel and AT&T. > The code was tested through an extensive beta program managed by Intel > (with more than 60 80386 beta sites)." > Mini-Micro Systems, January 1988, p 45 > https://archive.org/details/bitsavers_MiniMicroS_59292072/page/44/mode/2up > https://bitsavers.org/magazines/Mini-Micro_Systems/198801.pdf I'm not familiar with 386/ix so I'll have to let others comment here, though I do note that we're now slipping into 1988. AT&T sold rebadged Olivetti machines with SVR3 in 1987: > "AT&T 6386 WGS is today's only 80386-based system to take full advantage > of its 32-bit architecture" > https://bitsavers.org/pdf/att/6386_wgs/6386_WGS_Brochure.pdf This is fascinating, as it claims "concurrent running of both MS-DOS and true 32-bit UNIX System V programs." They were also serious about market positioning - the larger model could handle up to 64MB of RAM and two 135MB disks. I'd appreciate any further details on the exact operating system. > ISC work was also used by Microport: > "Microport Runtime System V/386 is based on a version of Unix for the > 80386 carried out by Interactive Technologies for AT&T and Intel." > Microport to Ship Version of Unix for 386 > InfoWorld, Volume 9, Issue 9, 2 Mar 1987, p 3 > https://books.google.com/books?id=1TAEAAAAMBAJ&pg=PA3 Also interesting; I wonder if the "capability to run multiple MS-DOS applications under Unix" was shipped in a functional form, and what relation it might or might not have had to what was running on the AT&T hardware. -Henry -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Sat Dec 7 23:09:17 2024 From: tuhs at tuhs.org (Arrigo Triulzi via TUHS) Date: Sat, 7 Dec 2024 14:09:17 +0100 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: Message-ID: > On 7 Dec 2024, at 13:34, Henry Bent wrote: > > Also interesting; I wonder if the "capability to run multiple MS-DOS applications under Unix" was shipped in a functional form, and what relation it might or might not have had to what was running on the AT&T hardware. I used to run an 80286 “Taiwan clone” (as we called them in Italy) with Xenix 286 and, later, its 386 version which was SCO by then (memory a bit fuzzy on when Xenix became SCO Xenix and the Unix) and I definitely could run MS-DOS programs on the 386 - you would use the dos command which mapped drives either to physical drives (i.e. A: was the floppy) or directories within the filesystem. It was often used for businesses which had their inventory on MS-DOS bespoke software but wanted to “multitask” so we had some very dirty code which would run the DOS program on the serial terminals writing to a “network” drive which was actually a directory in the Unix filesystem. In the meantime I was busy writing a migration layer which would allow us to compile the MS-DOS C code on Unix natively replacing, for example, code writing the pretty box characters on MS-DOS with curses equivalents. Fortunately the DB stuff was all C-ISAM for which a portable library existed. All of this was strictly with licenses found off the back of a truck, of course. Licensing bypass in Italy was a national sport. (this was all done using the VM86 extension which, incidentally, still exists in Intel chips… this opens a whole other story about forgotten Intel extensions lingering in 2024 cores which can be used nefariously). Arrigo From henry.r.bent at gmail.com Sat Dec 7 23:45:15 2024 From: henry.r.bent at gmail.com (Henry Bent) Date: Sat, 7 Dec 2024 08:45:15 -0500 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: Message-ID: On Sat, 7 Dec 2024 at 08:09, Arrigo Triulzi wrote: > > > On 7 Dec 2024, at 13:34, Henry Bent wrote: > > > > Also interesting; I wonder if the "capability to run multiple MS-DOS > applications under Unix" was shipped in a functional form, and what > relation it might or might not have had to what was running on the AT&T > hardware. > > I used to run an 80286 “Taiwan clone” (as we called them in Italy) with > Xenix 286 and, later, its 386 version which was SCO by then (memory a bit > fuzzy on when Xenix became SCO Xenix and the Unix) and I definitely could > run MS-DOS programs on the 386 - you would use the dos command which mapped > drives either to physical drives (i.e. A: was the floppy) or directories > within the filesystem. > > It was often used for businesses which had their inventory on MS-DOS > bespoke software but wanted to “multitask” so we had some very dirty code > which would run the DOS program on the serial terminals writing to a > “network” drive which was actually a directory in the Unix filesystem. > Interesting, thank you for the explanation. How was file locking handled for DOS programs? Did it have some sort of internal call to "share" or was there a more elegant method? -Henry -------------- next part -------------- An HTML attachment was scrubbed... URL: From davida at pobox.com Sun Dec 8 00:04:40 2024 From: davida at pobox.com (David Arnold) Date: Sun, 8 Dec 2024 01:04:40 +1100 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact In-Reply-To: References: Message-ID: <6835EFA5-A5C5-4F98-8F7F-9845BE83A504@pobox.com> > On 7 Dec 2024, at 23:11, Henry Bent wrote: > > The standard C compiler was a different beast, and I was not immediately able to recognize its lineage; it may have been developed from scratch. Derived from the Norcroft compiler Ihttp://www.codemist.co.uk/ncc/index.html d -------------- next part -------------- An HTML attachment was scrubbed... URL: From aek at bitsavers.org Sun Dec 8 00:09:28 2024 From: aek at bitsavers.org (Al Kossow) Date: Sat, 7 Dec 2024 06:09:28 -0800 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: Message-ID: On 12/7/24 5:09 AM, Arrigo Triulzi via TUHS wrote: > forgotten Intel extensions lingering in 2024 cores which can be used nefariously There has been an effort by Intel to expunge a lot of PC-isms from their CPUs and ASICs in recent years. I know someone who worked on that out of the architecture group in Oregon. WRT IBM's interest in the 386, OS/2 didn't release a 32 bit version until 2.0 in 1991, well into the 486 era (1989). https://www.quora.com/Why-did-IBMs-OS-2-project-lose-to-Microsoft-given-that-IBM-had-much-more-resources-than-Microsoft-at-that-time From henry.r.bent at gmail.com Sun Dec 8 01:08:00 2024 From: henry.r.bent at gmail.com (Henry Bent) Date: Sat, 7 Dec 2024 10:08:00 -0500 Subject: [TUHS] Xenix Distribution for PDP-11 Message-ID: Hello all, The recent discussion about Xenix has me curious: does any Xenix distribution for the PDP-11 survive? I've never seen one and it would be a fascinating historical artifact. That being said, I am not soliciting copyrighted software; I would be more than happy with a simple answer of, "yes, one has been archived." -Henry -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsg at jsg.id.au Sun Dec 8 01:27:59 2024 From: jsg at jsg.id.au (Jonathan Gray) Date: Sun, 8 Dec 2024 02:27:59 +1100 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: <26451.32253.73591.816292@dobie-old.ylee.org> Message-ID: On Sat, Dec 07, 2024 at 07:34:07AM -0500, Henry Bent wrote: > This is all very enlightening, thank you. Comments inline... > > On Sat, 7 Dec 2024 at 05:39, Jonathan Gray wrote: > > > The 1986 press release for the Deskpro 386 mentioned 386 Xenix > > was planned for 1987. > > https://www.latimes.com/archives/la-xpm-1986-09-10-fi-13177-story.html > > > I found https://www.tech-insider.org/unix/research/1987/0902.html which > includes the following quote: > > > "UNIX System V and the 80386 are a perfect technological match," said Bill > Gates, chairman of Microsoft Corporation, in remarks at AT&T's press > conference here. AT&T and Microsoft are developing a new version of UNIX > System V for the 80386 chip that will run XENIX System V as well as UNIX > System V applications. > > > This is September 1987, so perhaps Microsoft's abandonment of Xenix was not > as early as I had thought. Though this does imply that the Xenix port was > not ready at that point, and perhaps was ultimately abandoned by Microsoft. > > > > Intel and AT&T had ISC do a 386 port for SVR3. > > > > "The 386/ix is based on Release 3.0, which was developed by Interactive > > Systems Corp. Santa Monica, Calif., under contract to Intel and AT&T. > > The code was tested through an extensive beta program managed by Intel > > (with more than 60 80386 beta sites)." > > Mini-Micro Systems, January 1988, p 45 > > https://archive.org/details/bitsavers_MiniMicroS_59292072/page/44/mode/2up > > https://bitsavers.org/magazines/Mini-Micro_Systems/198801.pdf > > > I'm not familiar with 386/ix so I'll have to let others comment here, > though I do note that we're now slipping into 1988. ISC had it running well before then. I'm not sure when 386/ix was commercially available, 1987? "We keep reading articles that say there is no operating system that uses the full features of the 386, but we have had such an operating system functioning since last June." Betty Niimi, Interactive Systems Unix to Lead in 80386-Based Operating System Shipments InfoWorld, December 22, 1986, p 8 https://books.google.com/books?id=UjwEAAAAMBAJ&pg=PA8 > > AT&T sold rebadged Olivetti machines with SVR3 in 1987: > > "AT&T 6386 WGS is today's only 80386-based system to take full advantage > > of its 32-bit architecture" > > https://bitsavers.org/pdf/att/6386_wgs/6386_WGS_Brochure.pdf > > > This is fascinating, as it claims "concurrent running of both MS-DOS and > true 32-bit UNIX System V programs." They were also serious about market > positioning - the larger model could handle up to 64MB of RAM and two 135MB > disks. I'd appreciate any further details on the exact operating system. AT&T had 'Simul-Task 386' which was based on VP/ix from Phoenix Technologies Limited and Interactive Systems Corporation. VP/ix also appeared on the Sun 386i in 1988. > > > > ISC work was also used by Microport: > > "Microport Runtime System V/386 is based on a version of Unix for the > > 80386 carried out by Interactive Technologies for AT&T and Intel." > > Microport to Ship Version of Unix for 386 > > InfoWorld, Volume 9, Issue 9, 2 Mar 1987, p 3 > > https://books.google.com/books?id=1TAEAAAAMBAJ&pg=PA3 > > > Also interesting; I wonder if the "capability to run multiple MS-DOS > applications under Unix" was shipped in a functional form, and what > relation it might or might not have had to what was running on the AT&T > hardware. Microport sold 'Merge 386' from Locus Computing Corporation. Also available for AIX on PS/2 Model 80. AIX for PS/2 wasn't available till 1988. From jsg at jsg.id.au Sun Dec 8 02:02:11 2024 From: jsg at jsg.id.au (Jonathan Gray) Date: Sun, 8 Dec 2024 03:02:11 +1100 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: <26451.32253.73591.816292@dobie-old.ylee.org> Message-ID: On Sun, Dec 08, 2024 at 02:27:59AM +1100, Jonathan Gray wrote: > Microport sold 'Merge 386' from Locus Computing Corporation. > > Also available for AIX on PS/2 Model 80. AIX for PS/2 wasn't available > till 1988. It was planned for 1988, but was not available until 1989. From tuhs at tuhs.org Sun Dec 8 02:12:41 2024 From: tuhs at tuhs.org (Arrigo Triulzi via TUHS) Date: Sat, 7 Dec 2024 17:12:41 +0100 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: Message-ID: <91E629E9-2FCB-421A-8596-1D751656708E@alchemistowl.org> On 7 Dec 2024, at 14:45, Henry Bent wrote: > Interesting, thank you for the explanation. How was file locking handled for DOS programs? Did it have some sort of internal call to "share" or was there a more elegant method? Well, as the Unix filesystem was connected to MS-DOS as a “network drive” it had rudimentary opportunistic locking via the SMB protocol which I am not entirely sure actually translated to anything on the Unix side. There was often data corruption when writing from multiple MS-DOS sessions to the same file so the customer, who was particularly keen on the reading from multiple terminals more than the writing, simply decided that only one person could write into the inventory at one time. “Sneaker lock”? Anyway, the port to native Unix was achieved in a couple of months, this was the ‘80s, code was simple and you didn’t have three rows of obscure buttons, a menu on top, another one to the side, etc. to be able to write a letter… Once we moved to Unix C-ISAM implemented proper record-level locking in the library itself and the problem went away (except for the inevitable code changes to handle C-ISAM saying “can’t access the record as it is locked, try again later”). Arrigo From tuhs at tuhs.org Sun Dec 8 02:28:20 2024 From: tuhs at tuhs.org (Arrigo Triulzi via TUHS) Date: Sat, 7 Dec 2024 17:28:20 +0100 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: <91E629E9-2FCB-421A-8596-1D751656708E@alchemistowl.org> References: <91E629E9-2FCB-421A-8596-1D751656708E@alchemistowl.org> Message-ID: <119A33DC-0476-43E5-8BAD-A2F16280A7AC@alchemistowl.org> On 7 Dec 2024, at 17:12, Arrigo Triulzi via TUHS wrote: > Well, as the Unix filesystem was connected to MS-DOS as a “network drive” it had rudimentary opportunistic locking via the SMB protocol which I am not entirely sure actually translated to anything on the Unix side. There was often data corruption when writing from multiple MS-DOS sessions to the same file so the customer, who was particularly keen on the reading from multiple terminals more than the writing, simply decided that only one person could write into the inventory at one time. “Sneaker lock”? Correction: it cannot possibly be SMB, that came later - I haven’t the faintest idea of what the network protocol was called in MS-DOS, I just remember you had to load something in CONFIG.SYS on MS-DOS 4 or higher to make it work. Arrigo From imp at bsdimp.com Sun Dec 8 02:38:08 2024 From: imp at bsdimp.com (Warner Losh) Date: Sat, 7 Dec 2024 09:38:08 -0700 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: <119A33DC-0476-43E5-8BAD-A2F16280A7AC@alchemistowl.org> References: <91E629E9-2FCB-421A-8596-1D751656708E@alchemistowl.org> <119A33DC-0476-43E5-8BAD-A2F16280A7AC@alchemistowl.org> Message-ID: On Sat, Dec 7, 2024, 9:28 AM Arrigo Triulzi via TUHS wrote: > On 7 Dec 2024, at 17:12, Arrigo Triulzi via TUHS wrote: > > Well, as the Unix filesystem was connected to MS-DOS as a “network > drive” it had rudimentary opportunistic locking via the SMB protocol which > I am not entirely sure actually translated to anything on the Unix side. > There was often data corruption when writing from multiple MS-DOS sessions > to the same file so the customer, who was particularly keen on the reading > from multiple terminals more than the writing, simply decided that only one > person could write into the inventory at one time. “Sneaker lock”? > > Correction: it cannot possibly be SMB, that came later - I haven’t the > faintest idea of what the network protocol was called in MS-DOS, I just > remember you had to load something in CONFIG.SYS on MS-DOS 4 or higher to > make it work. > Yes. There were namespace hooks of a sort that one could hook into via a TSR... I wrote something for a DOS 3.1 system that had a few initial UNDOCUMENTED interfaces. I couldn't ever get anything to work beyond the init... Warner > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Sun Dec 8 02:41:28 2024 From: tuhs at tuhs.org (Arrigo Triulzi via TUHS) Date: Sat, 7 Dec 2024 17:41:28 +0100 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: <91E629E9-2FCB-421A-8596-1D751656708E@alchemistowl.org> <119A33DC-0476-43E5-8BAD-A2F16280A7AC@alchemistowl.org> Message-ID: <28A287D9-4D86-4DCF-9C23-32FD8D48CEEC@alchemistowl.org> On 7 Dec 2024, at 17:38, Warner Losh wrote: > Yes. There were namespace hooks of a sort that one could hook into via a TSR... I wrote something for a DOS 3.1 system that had a few initial UNDOCUMENTED interfaces. I couldn't ever get anything to work beyond the init... Yes, TSRs… the art of loading them in the correct sequence so that you did not make your machine unstable. Some we really quite useful, for example there was one which I seem to recall was called “Tornado” which allowed you to write linked and indexed notes, a bit like an early Hypercard? Then there were the ones hooking services to give you access to the RAM between 640kB and 1MB, the ones to page in & out memory > 1MB, oh the horrors… I was really rather pleased with my 386 running Unix! Arrigo From henry.r.bent at gmail.com Sun Dec 8 03:01:22 2024 From: henry.r.bent at gmail.com (Henry Bent) Date: Sat, 7 Dec 2024 12:01:22 -0500 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: <91E629E9-2FCB-421A-8596-1D751656708E@alchemistowl.org> References: <91E629E9-2FCB-421A-8596-1D751656708E@alchemistowl.org> Message-ID: On Sat, 7 Dec 2024 at 11:12, Arrigo Triulzi wrote: > On 7 Dec 2024, at 14:45, Henry Bent wrote: > > Interesting, thank you for the explanation. How was file locking > handled for DOS programs? Did it have some sort of internal call to > "share" or was there a more elegant method? > > Well, as the Unix filesystem was connected to MS-DOS as a “network drive” > it had rudimentary opportunistic locking via the SMB protocol which I am > not entirely sure actually translated to anything on the Unix side. There > was often data corruption when writing from multiple MS-DOS sessions to the > same file so the customer, who was particularly keen on the reading from > multiple terminals more than the writing, simply decided that only one > person could write into the inventory at one time. “Sneaker lock”? > Sadly, that's the answer I was expecting - the locking didn't really work in practice. That might go some way towards explaining why this concept of multiple DOS sessions under UNIX didn't really have widespread adoption. There were always all sorts of "DOS under UNIX" ideas, from these early concepts through all the way to Sun's physical PC boards, but none of them ever really seemed to gain significant traction. The only connecting concept seems to be that DOS just wasn't meant to be a multi-user OS, and certainly not a networked one. -Henry -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Sun Dec 8 04:15:55 2024 From: tuhs at tuhs.org (Arrigo Triulzi via TUHS) Date: Sat, 7 Dec 2024 19:15:55 +0100 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: <91E629E9-2FCB-421A-8596-1D751656708E@alchemistowl.org> Message-ID: <3FD82849-BDEB-4AED-A805-44D2FE45918E@alchemistowl.org> On 7 Dec 2024, at 18:01, Henry Bent wrote: > Sadly, that's the answer I was expecting - the locking didn't really work in practice. That might go some way towards explaining why this concept of multiple DOS sessions under UNIX didn't really have widespread adoption. I cannot honestly see how it could be made to work - locking, even in Unix, was very much a case of “let’s hope we all use the same function call” and, in the specific case of MS-DOS, was hardly necessary since it was a single-user OS running a single application at a time. The arrival of networking probably introduced locking to MS-DOS in the same way hopeful way. There was no real expectation on our part that the DB would not be corrupted by multiple MS-DOS sessions writing to it, even if using C-ISAM, because the assumption would have always been “one machine, one user, one task”. Fortunately the port to Unix fixed this because C-ISAM was conscious of the concept of multiple accesses to the database. > There were always all sorts of "DOS under UNIX" ideas, from these early concepts through all the way to Sun's physical PC boards, but none of them ever really seemed to gain significant traction. The only connecting concept seems to be that DOS just wasn't meant to be a multi-user OS, and certainly not a networked one. Even on the i286 there were attempts at having MS-DOS running within a multi-user OS. Remember that OS/2 on the 80286 used a triple fault to get you back into real mode to execute an MS-DOS program. "Running MS-DOS” was really important in those days and a lot of effort was expended trying to get that to work in some way[1]. Arrigo [1] I am a (very) guilty party for flooring my CPUs and making fans sound like 747s by running Simcity 2000 and Railroad Tycoon in dosbox on Linux. From ron at ronnatalie.com Sun Dec 8 06:38:02 2024 From: ron at ronnatalie.com (Ron Natalie) Date: Sat, 07 Dec 2024 20:38:02 +0000 Subject: [TUHS] Pipes (was Re: After 50 years, what has the Impact of Unix been?) In-Reply-To: References: <568FD44F-01FB-441B-846B-7D42C3A8E1FB@canb.auug.org.au> <20241205030843.8552FAB1EDA5@ary.qy> <32bf4263-277d-8c8b-6bf7-e33df04a2c3c@taugh.com> Message-ID: NO, the 11/23 has an MMU and was perflectly capable of running UNIX. The 23+ had the extra address lines allowing it to address 4 MB like the 11/70. ------ Original Message ------ >From "Adam Thornton" To "Ron Natalie" ; "The Eunuchs Hysterical Society" Date 12/5/2024 9:29:24 PM Subject Re: [TUHS] Re: Pipes (was Re: After 50 years, what has the Impact of Unix been?) >/23+, no? The basic 23 doesn't have an MMU, I'm pretty sure. > >On Thu, Dec 5, 2024 at 11:58 AM Ron Natalie wrote: >>I remember on MiniUnix (a stripped down kernel that would run on >>PDP11’s >>without memory management, in our case 11/03, 11/20, and an older >>11/40) >>there were no kernel pipes (and limited process concurrency). >>The shell used the aforementioned emulation of pipe by just running >>the >>output of one process into a temporary file subsequently loaded as >>stdin >>of the other. >> >>We kind of put all that to bed when the 11/23s came out and we could >>run >>our full up kernel on the micros. >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jnc at mercury.lcs.mit.edu Sun Dec 8 07:06:15 2024 From: jnc at mercury.lcs.mit.edu (Noel Chiappa) Date: Sat, 7 Dec 2024 16:06:15 -0500 (EST) Subject: [TUHS] A re-analysis of some DECtapes from dmr Message-ID: <20241207210615.0CBA818C079@mercury.lcs.mit.edu> > From: Yufeng Gao > This version of UNIX appears to be between V1 and V2 It's too bad it's not 'between V2 and V3', since the UNIX kernels V2 and V3 are missing. (V4 is the earlist one that we have, after the V2 and onward gap.) So, depending on how far from V1 (which we have) it is, it _might_ be worth dis-assenbling. It would be particularly worth having it if it suppports the KS11, since almost no documentation of that has survived; code that uses it would allow re-creation of a programming manual for it. > From: Lars Brinkhoff > Is the kernel for a PDP-11/20 or /45? I don't think it can be the /45; as of V2, the /45 was apparently expected in the near future, and this was before the V2. Noel From tuhs at tuhs.org Sun Dec 8 11:01:28 2024 From: tuhs at tuhs.org (Warren Toomey via TUHS) Date: Sun, 8 Dec 2024 11:01:28 +1000 Subject: [TUHS] A re-analysis of some DECtapes from dmr In-Reply-To: <7wr06jsur9.fsf@junk.nocrew.org> References: <7wr06jsur9.fsf@junk.nocrew.org> Message-ID: On Sat, Dec 07, 2024 at 09:32:58AM +0000, Lars Brinkhoff wrote: > Is the kernel for a PDP-11/20 or /45? If /20, does it use the KS11 for > memory mapping? Lars, it's the same kernel as in this project: https://github.com/DoctorWkt/unix-jun72 Here's part of the SimH config: set cpu 11/20 set cpu 32K set rk0 enabled set rf enabled set tc enabled set tc enabled set rf enabled set ke enabled set dci en Cheers, Warren From tuhs at tuhs.org Mon Dec 9 17:14:17 2024 From: tuhs at tuhs.org (Arno Griffioen via TUHS) Date: Mon, 9 Dec 2024 08:14:17 +0100 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: <26451.32253.73591.816292@dobie-old.ylee.org> Message-ID: On Sat, Dec 07, 2024 at 07:34:07AM -0500, Henry Bent wrote: > "UNIX System V and the 80386 are a perfect technological match," said Bill > Gates, chairman of Microsoft Corporation, in remarks at AT&T's press conference > here. AT&T and Microsoft are developing a new version of UNIX System V for the > 80386 chip that will run XENIX System V as well as UNIX System V applications. > > This is September 1987, so perhaps Microsoft's abandonment of Xenix was not as > early as I had thought.  Though this does imply that the Xenix port was not > ready at that point, and perhaps was ultimately abandoned by Microsoft. https://www.landley.net/history/mirror/unix/scohistory.html states that: "1987 - SCO ships SCO XENIX 386, the first 32-bit operating system (and first UNIX System) for Intel 386 processor-based systems." So it looks like XENIX development had been already stopped at MS and taken on/over by SCO in the meantime. Matches up with the WIKI doc on Xenix and the transfer of it's ownership from MS to SCO: https://en.wikipedia.org/wiki/Xenix#:~:text=SCO's%20Xenix%20System%20V%2F386,both%20Xenix%20and%20SCO%20Unix. So it seems around 1985/6 MS stopped any real Xenix development and got out of the UNIX market, after which SCO went on with it. Side-note... Compaq servers running SCO UNIX versions were *very* popular in the small to medium business area here from the late 80's onward. Lots of 'productivity' software like administrative/financial/etc. software on SCO that allowed many people to work concurrently using one or a few servers and either terminals or some sort of networking. (Ahhhhh... The days of the network-wars with oodles of competing more and lesser known network types.. Messy and convoluted, but quite fun... :) ) The fact that it was UNIX underneath was totally irrelevant to most end customers though. They just got/ran it to run their business and it worked much better than any MS-based solution until the early 2000s. Used to be a large part of my work during the early 90's to maintain, upgrade and fix such systems as the end customers usually had no technical people onsite anyway. Bye, Arno. From woods at robohack.ca Wed Dec 11 08:41:30 2024 From: woods at robohack.ca (Greg A. Woods) Date: Tue, 10 Dec 2024 14:41:30 -0800 Subject: [TUHS] Interesting post about Microsoft and UNIX In-Reply-To: References: <26451.32253.73591.816292@dobie-old.ylee.org> Message-ID: At Fri, 6 Dec 2024 20:55:36 -0700, Marc Rochkind wrote: Subject: [TUHS] Re: Interesting post about Microsoft and UNIX > > I'm sure Compaq was thinking ahead. They would be very aware of Microsoft's > Windows plans. Buyers also think ahead, especially corporate buyers, who > are the real customers. I was working on contracts at Canadian Pacific Railway at about that time. I'm pretty sure CP would have been a big customer for 386 machines. They were an early adopter of PCs in general for a larger corporation (CP was one of the first "paperless" companies in Canada, with PCs on every desktop and being pushed out to big customers to interface via 3270 emulation cards and custom software with their mainframe systems for ordering train cars, etc. Meanwhile many similar- sized companies with mainframes were still all just using plain old 3270 terminals (i.e. without custom interface software).) At the time of the debut of the 386 we were building a new track control system for the Broadview subdivision and it was being built with Intel 286 systems running Xenix. I think they may have been Compaq hardware, though earlier MS-DOS projects I worked on at CP were using Olivetti PCs, though CP also certainly had PCs from Compaq and IBM as well. I was doing device drivers (including one for a Matrox graphics card), various support libraries such as an IPC framework, and general toolchain and OS support for the project. I remember putting up a poster from Intel of the i386 chip in my cubicle to remind colleagues of the exciting 32-bit things soon to come. I don't remember having any worries that there would be any problem with getting a 386 version of Xenix or some other Unix. -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From crossd at gmail.com Sat Dec 14 00:28:38 2024 From: crossd at gmail.com (Dan Cross) Date: Fri, 13 Dec 2024 09:28:38 -0500 Subject: [TUHS] BSD talk program? Message-ID: I'm curious if anyone has any history they can share about the BSD "talk" program. I was fond of this back when it was still (relatively) common, but given the way it's architected I definitely see why it fell out of use as the Internet grew. Still, does anybody know what the history behind it is? Initially, I thought it was written by Mike Karels, but that was just my speculation from SCCS spelunking, and looking at the sources from 4.2, I see RCS header strings that indicate it was written by "moore" (Peter Moore?). talk.c says, "Written by Kipp Hickman". It seems to have arrived pretty early on with respect to the introduction of TCP/IP in BSD: the README alludes to some things coming up in 4.1c. Clem, you seem to have had a hand in it, and are credited (along with Peter Moore) for making it work on 4.1a. So I guess the question is, what was the motivation? Was it just to have a more pleasing user-to-user communications experience, or was discussion across the network an explicit goal? There's a note in talk.c ("Modified to run between hosts by Peter Moore, 8/19/82") that suggests this wasn't the original intent. Who thought up the character-at-a-time display mode? Thanks for any insights. - Dan C. From clemc at ccc.com Sat Dec 14 01:14:06 2024 From: clemc at ccc.com (Clem Cole) Date: Fri, 13 Dec 2024 10:14:06 -0500 Subject: [TUHS] BSD talk program? In-Reply-To: References: Message-ID: Yes -- I can give this history. Kipp wrote an early version for 4.1BSD - but it is not the version in the releases. It ran on Ernie and did not do as much. I had used a different program on the PDP-10's and the ARPANET and I started over when Joy added sockets for 4.1A. I also made the infamous use of vax integers instead of network integers (and I knew better - but really did not think about until a few years later when I was at Masscomp and compiled it for the 68000 -- ugh). That version still had a couple of bugs in it (i.e. hung in the 4.1A networking code occasionally), but worked well enough on the CAD systems. I went away to a USENIX conference and while I was gone, my officemate Peter (Moore) took my code and fixed the problem, plus he put it into RCS. I gave that to Sam and that's the version that went out in 4.1C and beyond. Clem ᐧ ᐧ On Fri, Dec 13, 2024 at 9:29 AM Dan Cross wrote: > I'm curious if anyone has any history they can share about the BSD > "talk" program. > > I was fond of this back when it was still (relatively) common, but > given the way it's architected I definitely see why it fell out of use > as the Internet grew. Still, does anybody know what the history behind > it is? Initially, I thought it was written by Mike Karels, but that > was just my speculation from SCCS spelunking, and looking at the > sources from 4.2, I see RCS header strings that indicate it was > written by "moore" (Peter Moore?). talk.c says, "Written by Kipp > Hickman". > > It seems to have arrived pretty early on with respect to the > introduction of TCP/IP in BSD: the README alludes to some things > coming up in 4.1c. Clem, you seem to have had a hand in it, and are > credited (along with Peter Moore) for making it work on 4.1a. > > So I guess the question is, what was the motivation? Was it just to > have a more pleasing user-to-user communications experience, or was > discussion across the network an explicit goal? There's a note in > talk.c ("Modified to run between hosts by Peter Moore, 8/19/82") that > suggests this wasn't the original intent. Who thought up the > character-at-a-time display mode? > > Thanks for any insights. > > - Dan C. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Sat Dec 14 01:22:22 2024 From: clemc at ccc.com (Clem Cole) Date: Fri, 13 Dec 2024 10:22:22 -0500 Subject: [TUHS] BSD talk program? In-Reply-To: References: Message-ID: As for the motivation -- it was simple. UCB is on a hill. I lived at the base of hill and I only wanted to walk up it once a day. Our office was a big pool of about 20 of us next to the CAD machine room on the second floor of Cory Hall. Somebody was usually in the office most nights, but not everyone. We all had modems and terminals at home, but only one phone line. We had 3 Vaxes in the CAD group, plus my Array Processor. So I wanted to be able to ask someone like Peter or TQ to reset the AP for me if I hosed it when I was working from home when I was debugging it. Plus the obvious social aspects -- "hey you want go get a Pizza/Beer etc..." But since we might be working on a different system, Kipps' hack was useless. ᐧ ᐧ ᐧ ᐧ On Fri, Dec 13, 2024 at 10:14 AM Clem Cole wrote: > Yes -- I can give this history. > Kipp wrote an early version for 4.1BSD - but it is not the version in the > releases. It ran on Ernie and did not do as much. > I had used a different program on the PDP-10's and the ARPANET and I > started over when Joy added sockets for 4.1A. I also made the infamous use > of vax integers instead of network integers (and I knew better - but really > did not think about until a few years later when I was at Masscomp and > compiled it for the 68000 -- ugh). That version still had a couple of bugs > in it (i.e. hung in the 4.1A networking code occasionally), but worked well > enough on the CAD systems. I went away to a USENIX conference and while I > was gone, my officemate Peter (Moore) took my code and fixed the problem, > plus he put it into RCS. I gave that to Sam and that's the version that > went out in 4.1C and beyond. > > Clem > > > ᐧ > ᐧ > > On Fri, Dec 13, 2024 at 9:29 AM Dan Cross wrote: > >> I'm curious if anyone has any history they can share about the BSD >> "talk" program. >> >> I was fond of this back when it was still (relatively) common, but >> given the way it's architected I definitely see why it fell out of use >> as the Internet grew. Still, does anybody know what the history behind >> it is? Initially, I thought it was written by Mike Karels, but that >> was just my speculation from SCCS spelunking, and looking at the >> sources from 4.2, I see RCS header strings that indicate it was >> written by "moore" (Peter Moore?). talk.c says, "Written by Kipp >> Hickman". >> >> It seems to have arrived pretty early on with respect to the >> introduction of TCP/IP in BSD: the README alludes to some things >> coming up in 4.1c. Clem, you seem to have had a hand in it, and are >> credited (along with Peter Moore) for making it work on 4.1a. >> >> So I guess the question is, what was the motivation? Was it just to >> have a more pleasing user-to-user communications experience, or was >> discussion across the network an explicit goal? There's a note in >> talk.c ("Modified to run between hosts by Peter Moore, 8/19/82") that >> suggests this wasn't the original intent. Who thought up the >> character-at-a-time display mode? >> >> Thanks for any insights. >> >> - Dan C. >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Sat Dec 14 01:53:36 2024 From: lm at mcvoy.com (Larry McVoy) Date: Fri, 13 Dec 2024 07:53:36 -0800 Subject: [TUHS] BSD talk program? In-Reply-To: References: Message-ID: <20241213155336.GV11590@mcvoy.com> I loved talk when CS was running BSD on a VAX. You could see who was on and talk them. Very handy and it was sort of social. It's crazy how things were back then, open ports listening for all sorts of things. I think we were pretty unaware of how nasty the internet would get. On Fri, Dec 13, 2024 at 10:22:22AM -0500, Clem Cole wrote: > As for the motivation -- it was simple. UCB is on a hill. I lived at the > base of hill and I only wanted to walk up it once a day. Our office was a > big pool of about 20 of us next to the CAD machine room on the second floor > of Cory Hall. Somebody was usually in the office most nights, but not > everyone. We all had modems and terminals at home, but only one phone > line. We had 3 Vaxes in the CAD group, plus my Array Processor. So I > wanted to be able to ask someone like Peter or TQ to reset the AP for me if > I hosed it when I was working from home when I was debugging it. Plus > the obvious social aspects -- "hey you want go get a Pizza/Beer etc..." > But since we might be working on a different system, Kipps' hack was > useless. > ??? > ??? > ??? > ??? > > On Fri, Dec 13, 2024 at 10:14???AM Clem Cole wrote: > > > Yes -- I can give this history. > > Kipp wrote an early version for 4.1BSD - but it is not the version in the > > releases. It ran on Ernie and did not do as much. > > I had used a different program on the PDP-10's and the ARPANET and I > > started over when Joy added sockets for 4.1A. I also made the infamous use > > of vax integers instead of network integers (and I knew better - but really > > did not think about until a few years later when I was at Masscomp and > > compiled it for the 68000 -- ugh). That version still had a couple of bugs > > in it (i.e. hung in the 4.1A networking code occasionally), but worked well > > enough on the CAD systems. I went away to a USENIX conference and while I > > was gone, my officemate Peter (Moore) took my code and fixed the problem, > > plus he put it into RCS. I gave that to Sam and that's the version that > > went out in 4.1C and beyond. > > > > Clem > > > > > > ??? > > ??? > > > > On Fri, Dec 13, 2024 at 9:29???AM Dan Cross wrote: > > > >> I'm curious if anyone has any history they can share about the BSD > >> "talk" program. > >> > >> I was fond of this back when it was still (relatively) common, but > >> given the way it's architected I definitely see why it fell out of use > >> as the Internet grew. Still, does anybody know what the history behind > >> it is? Initially, I thought it was written by Mike Karels, but that > >> was just my speculation from SCCS spelunking, and looking at the > >> sources from 4.2, I see RCS header strings that indicate it was > >> written by "moore" (Peter Moore?). talk.c says, "Written by Kipp > >> Hickman". > >> > >> It seems to have arrived pretty early on with respect to the > >> introduction of TCP/IP in BSD: the README alludes to some things > >> coming up in 4.1c. Clem, you seem to have had a hand in it, and are > >> credited (along with Peter Moore) for making it work on 4.1a. > >> > >> So I guess the question is, what was the motivation? Was it just to > >> have a more pleasing user-to-user communications experience, or was > >> discussion across the network an explicit goal? There's a note in > >> talk.c ("Modified to run between hosts by Peter Moore, 8/19/82") that > >> suggests this wasn't the original intent. Who thought up the > >> character-at-a-time display mode? > >> > >> Thanks for any insights. > >> > >> - Dan C. > >> > > -- --- Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From imp at bsdimp.com Sat Dec 14 02:00:37 2024 From: imp at bsdimp.com (Warner Losh) Date: Fri, 13 Dec 2024 09:00:37 -0700 Subject: [TUHS] BSD talk program? In-Reply-To: <20241213155336.GV11590@mcvoy.com> References: <20241213155336.GV11590@mcvoy.com> Message-ID: On Fri, Dec 13, 2024, 8:53 AM Larry McVoy wrote: > I loved talk when CS was running BSD on a VAX. You could see who was on > and talk them. Very handy and it was sort of social. > > It's crazy how things were back then, open ports listening for all sorts > of things. I think we were pretty unaware of how nasty the internet would > get. And the talk protocol just passed random addresses around in binary format to make the connection and the client and server just trusted each other. Also, finger could see who you were talking to, like privacy didn't matter. Ah, Simpler times. Warner On Fri, Dec 13, 2024 at 10:22:22AM -0500, Clem Cole wrote: > > As for the motivation -- it was simple. UCB is on a hill. I lived at > the > > base of hill and I only wanted to walk up it once a day. Our office > was a > > big pool of about 20 of us next to the CAD machine room on the second > floor > > of Cory Hall. Somebody was usually in the office most nights, but not > > everyone. We all had modems and terminals at home, but only one phone > > line. We had 3 Vaxes in the CAD group, plus my Array Processor. So I > > wanted to be able to ask someone like Peter or TQ to reset the AP for me > if > > I hosed it when I was working from home when I was debugging it. Plus > > the obvious social aspects -- "hey you want go get a Pizza/Beer etc..." > > But since we might be working on a different system, Kipps' hack was > > useless. > > ??? > > ??? > > ??? > > ??? > > > > On Fri, Dec 13, 2024 at 10:14???AM Clem Cole wrote: > > > > > Yes -- I can give this history. > > > Kipp wrote an early version for 4.1BSD - but it is not the version in > the > > > releases. It ran on Ernie and did not do as much. > > > I had used a different program on the PDP-10's and the ARPANET and I > > > started over when Joy added sockets for 4.1A. I also made the infamous > use > > > of vax integers instead of network integers (and I knew better - but > really > > > did not think about until a few years later when I was at Masscomp and > > > compiled it for the 68000 -- ugh). That version still had a couple of > bugs > > > in it (i.e. hung in the 4.1A networking code occasionally), but worked > well > > > enough on the CAD systems. I went away to a USENIX conference and > while I > > > was gone, my officemate Peter (Moore) took my code and fixed the > problem, > > > plus he put it into RCS. I gave that to Sam and that's the version > that > > > went out in 4.1C and beyond. > > > > > > Clem > > > > > > > > > ??? > > > ??? > > > > > > On Fri, Dec 13, 2024 at 9:29???AM Dan Cross wrote: > > > > > >> I'm curious if anyone has any history they can share about the BSD > > >> "talk" program. > > >> > > >> I was fond of this back when it was still (relatively) common, but > > >> given the way it's architected I definitely see why it fell out of use > > >> as the Internet grew. Still, does anybody know what the history behind > > >> it is? Initially, I thought it was written by Mike Karels, but that > > >> was just my speculation from SCCS spelunking, and looking at the > > >> sources from 4.2, I see RCS header strings that indicate it was > > >> written by "moore" (Peter Moore?). talk.c says, "Written by Kipp > > >> Hickman". > > >> > > >> It seems to have arrived pretty early on with respect to the > > >> introduction of TCP/IP in BSD: the README alludes to some things > > >> coming up in 4.1c. Clem, you seem to have had a hand in it, and are > > >> credited (along with Peter Moore) for making it work on 4.1a. > > >> > > >> So I guess the question is, what was the motivation? Was it just to > > >> have a more pleasing user-to-user communications experience, or was > > >> discussion across the network an explicit goal? There's a note in > > >> talk.c ("Modified to run between hosts by Peter Moore, 8/19/82") that > > >> suggests this wasn't the original intent. Who thought up the > > >> character-at-a-time display mode? > > >> > > >> Thanks for any insights. > > >> > > >> - Dan C. > > >> > > > > > -- > --- > Larry McVoy Retired to fishing > http://www.mcvoy.com/lm/boat > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rich.salz at gmail.com Sat Dec 14 02:04:40 2024 From: rich.salz at gmail.com (Rich Salz) Date: Fri, 13 Dec 2024 11:04:40 -0500 Subject: [TUHS] BSD talk program? In-Reply-To: <20241213155336.GV11590@mcvoy.com> References: <20241213155336.GV11590@mcvoy.com> Message-ID: On Fri, Dec 13, 2024 at 11:03 AM Larry McVoy wrote: > I loved talk when CS was running BSD on a VAX. You could see who was on > and talk them. Very handy and it was sort of social. > Talk was okay, but hunt was great! :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrochkind at gmail.com Sat Dec 14 02:52:28 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Fri, 13 Dec 2024 09:52:28 -0700 Subject: [TUHS] SCCS roach motel Message-ID: IEEE Transactions on Software Engineering has asked me to write a retrospective on the influence of SCCS over the last 50 years, as my SCCS paper was published in 1975. They consider it one of the most influential papers from TSE's first decade. There's a funny quote from Ken Thompson that circulates from time-to-time: "SCCS, the source motel! Programs check in and never check out!" But nobody seems to know what it means exactly. As part of my research, I asked Ken what the quote meant, sunce I wanted to include it. He explained that it refers to SCCS storing binary data in its repository file, preventing UNIX text tools from operating on the file. Of course, this is only one of SCCS's many weaknesses. If you have anything funny about any of the others, post it here. I already have all the boring usual stuff (e.g., long-term locks, file-oriented, no merging). Marc Rochkind mrochkind.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ron at ronnatalie.com Sat Dec 14 02:58:22 2024 From: ron at ronnatalie.com (Ronald Natalie) Date: Fri, 13 Dec 2024 16:58:22 +0000 Subject: [TUHS] BSD talk program? In-Reply-To: <20241213155336.GV11590@mcvoy.com> References: <20241213155336.GV11590@mcvoy.com> Message-ID: Anybody remember jtalk? It was a hacked version of talk that ran your input through the “jive filter”. Slap ma fro! ------ Original Message ------ >From "Larry McVoy" To "Clem Cole" Cc "TUHS" Date 12/13/2024 10:53:36 AM Subject [TUHS] Re: BSD talk program? >I loved talk when CS was running BSD on a VAX. You could see who was on >and talk them. Very handy and it was sort of social. > >It's crazy how things were back then, open ports listening for all sorts >of things. I think we were pretty unaware of how nasty the internet would >get. > >On Fri, Dec 13, 2024 at 10:22:22AM -0500, Clem Cole wrote: >> As for the motivation -- it was simple. UCB is on a hill. I lived at the >> base of hill and I only wanted to walk up it once a day. Our office was a >> big pool of about 20 of us next to the CAD machine room on the second floor >> of Cory Hall. Somebody was usually in the office most nights, but not >> everyone. We all had modems and terminals at home, but only one phone >> line. We had 3 Vaxes in the CAD group, plus my Array Processor. So I >> wanted to be able to ask someone like Peter or TQ to reset the AP for me if >> I hosed it when I was working from home when I was debugging it. Plus >> the obvious social aspects -- "hey you want go get a Pizza/Beer etc..." >> But since we might be working on a different system, Kipps' hack was >> useless. >> ??? >> ??? >> ??? >> ??? >> >> On Fri, Dec 13, 2024 at 10:14???AM Clem Cole wrote: >> >> > Yes -- I can give this history. >> > Kipp wrote an early version for 4.1BSD - but it is not the version in the >> > releases. It ran on Ernie and did not do as much. >> > I had used a different program on the PDP-10's and the ARPANET and I >> > started over when Joy added sockets for 4.1A. I also made the infamous use >> > of vax integers instead of network integers (and I knew better - but really >> > did not think about until a few years later when I was at Masscomp and >> > compiled it for the 68000 -- ugh). That version still had a couple of bugs >> > in it (i.e. hung in the 4.1A networking code occasionally), but worked well >> > enough on the CAD systems. I went away to a USENIX conference and while I >> > was gone, my officemate Peter (Moore) took my code and fixed the problem, >> > plus he put it into RCS. I gave that to Sam and that's the version that >> > went out in 4.1C and beyond. >> > >> > Clem >> > >> > >> > ??? >> > ??? >> > >> > On Fri, Dec 13, 2024 at 9:29???AM Dan Cross wrote: >> > >> >> I'm curious if anyone has any history they can share about the BSD >> >> "talk" program. >> >> >> >> I was fond of this back when it was still (relatively) common, but >> >> given the way it's architected I definitely see why it fell out of use >> >> as the Internet grew. Still, does anybody know what the history behind >> >> it is? Initially, I thought it was written by Mike Karels, but that >> >> was just my speculation from SCCS spelunking, and looking at the >> >> sources from 4.2, I see RCS header strings that indicate it was >> >> written by "moore" (Peter Moore?). talk.c says, "Written by Kipp >> >> Hickman". >> >> >> >> It seems to have arrived pretty early on with respect to the >> >> introduction of TCP/IP in BSD: the README alludes to some things >> >> coming up in 4.1c. Clem, you seem to have had a hand in it, and are >> >> credited (along with Peter Moore) for making it work on 4.1a. >> >> >> >> So I guess the question is, what was the motivation? Was it just to >> >> have a more pleasing user-to-user communications experience, or was >> >> discussion across the network an explicit goal? There's a note in >> >> talk.c ("Modified to run between hosts by Peter Moore, 8/19/82") that >> >> suggests this wasn't the original intent. Who thought up the >> >> character-at-a-time display mode? >> >> >> >> Thanks for any insights. >> >> >> >> - Dan C. >> >> >> > > >-- >--- >Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From mrochkind at gmail.com Sat Dec 14 03:58:37 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Fri, 13 Dec 2024 10:58:37 -0700 Subject: [TUHS] SCCS roach motel In-Reply-To: References: Message-ID: On Fri, Dec 13, 2024 at 10:03 AM Mark Seiden wrote: > (I know there is a special place in hell for those who explain a joke, > but, you asked…) > > it’s just an allusion to the Black Flag Roach Motel product (still being > produced) > which has a trademark on the phrase “Roaches Check in… But they Don’t > Check Out”. > > Yeah, I knew that much. My question to Ken was about what this was saying about SCCS. Marc -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Sat Dec 14 04:06:49 2024 From: lm at mcvoy.com (Larry McVoy) Date: Fri, 13 Dec 2024 10:06:49 -0800 Subject: [TUHS] SCCS roach motel In-Reply-To: References: Message-ID: <20241213180649.GW11590@mcvoy.com> On Fri, Dec 13, 2024 at 09:52:28AM -0700, Marc Rochkind wrote: > IEEE Transactions on Software Engineering has asked me to write a > retrospective on the influence of SCCS over the last 50 years, as my SCCS > paper was published in 1975. They consider it one of the most influential > papers from TSE's first decade. > > There's a funny quote from Ken Thompson that circulates from time-to-time: > > "SCCS, the source motel! Programs check in and never check out!" > > But nobody seems to know what it means exactly. As part of my research, I > asked Ken what the quote meant, sunce I wanted to include it. He explained > that it refers to SCCS storing binary data in its repository file, > preventing UNIX text tools from operating on the file. > > Of course, this is only one of SCCS's many weaknesses. If you have anything > funny about any of the others, post it here. I already have all the boring > usual stuff (e.g., long-term locks, file-oriented, no merging). Warning, I know more about SCCS than the average person, I've reimplemented it from scratch and then built BitKeeper on top of an extended SCCS file format. So lots of info coming. Rick Smith and Wayne Scott know as much as I do, Rick knows more, he joined me and promptly started fixing my SCCS implementation. So far as I know, there is not a more knowledgable person that Rick when it comes to weave file formats. SCCS's strength is the weave format. It's largely not understood, even by other people working in source management. Here's the benefit of that weave (if people use it, which, other than BitKeeper, they don't, looking at you, Clearcase, you had a weave and didn't use it): SCCS can pass merge data by reference, everyone else copies the data. SCCS is a set based system. Each node has a revision number, like 1.5, but because SCCS, unlike RCS, limited the revisions to at most 4 fields, like 1.5.1.1, it is _impossible_ to build the history graph from the revisions, you can in simple graphs but as soon as you branch from a branch, all bets are off. The graph is built from what BitKeeper called serial numbers. Each node in the graph has at least 2 serials, one that names that node in the graph, and one that is the parent. So if I have a file with 5 revisions in straight line history, the internal stuff will look something like rev me parent 1.5 5 4 1.4 4 3 1.3 3 2 1.2 2 1 1.1 1 0 So what's the set? Pretty simple for straight line history, you walk the history from the rev that you want, adding the "me" serial and going to the parent, repeat until the parent is 0. Suppose you branch from rev 1.3. rev me parent 1.3.1.1 6 3 1.5 5 4 1.4 4 3 1.3 3 2 ... See that 1.3.1.1 is me: 6 and parent: 3. So if I were building the set for 1.3.1.1, it becomes 6, then go to parent 3, 2, 1, skipping over 5 and 4. If you understand that, you are starting to understand the set and how it is constructed. Did you know you can have an argument in the revision history without adding anything to the data part? SCCS has the ability to include and/or exclude serials as part of a delta. Lets say Marc looked at my 1.5 and thought it was garbage. He can exclude it from the set like so: rev me parent include exclude 1.6 7 5 0 5 1.3.1.1 6 3 1.5 5 4 1.4 4 3 1.3 3 2 ... That doesn't change the data part of the file AT ALL, it's just saying Marc doesn't want anyone to see the 1.5 changes. To understand that, you need to know how SCCS checks out a file. And you need to know how the data is stored. Which is in a weave. RCS, and pretty much everything that followed it, doesn't use a weave at all. RCS stores the most recent version of the file as a complete copy of the checked out file. Then each delta working backwards up the trunk is a patch, what diff produces. Think about what that means for working on a branch. You have to start with the most recent version of the file, apply backward patches to go to earlier versions all the way back to the branch point, then apply forward patches to work your way to tip of the branch. Ask Dave Miller how pleasant it is to work on gcc on a branch. It's crazy slow and painful. So how does SCCS do it? Lets say the first version of a file is 1 2 3 4 5 The data portion of the history file will look like: ^AI 1 1 2 3 4 5 ^AE 1 SCCS used ^A at the beginning of a line to mean "this is metadata for SCCS". ^AI is an insert, ^AD is a delete, and insert/delete are paired with a ^AE which means end. The number after is the serial. So that weave says "If serial 1 is in your set, everything after ^AI 1 is part of that set until you hit the matching ^AE 1. Lets say the 2nd version is 1 2 serial 2 added this 3 4 Notice that serial 2 deleted what was line 5. ^AI 1 1 2 ^AI 2 serial 2 added this ^AE 2 3 4 ^AD 2 5 ^AE 2 ^AE 1 So now we can start to see how you walk the weave. If I'm trying to check out 1.1 aka serial 1, I build a set that has only '1' in the set. I hit the ^AI 1 see that I have 1 in my set, so I'm now in print mode, which means print each data line. I hit ^AI 2, that's not in my set, so I'm now in skip mode. And I skip the stuff inserted by serial 2. I see the ^AE 2 and I revert back to print mode. I get to ^AD 2, 2 is NOT in my set, so I stay in print mode. Etc. Let's make a branch, 1.1.1.1, with lots of data. 1 2 3 branch line 1 branch line 2 ... branch line 10000 4 5 ^AI 1 1 2 ^AI 2 serial 2 added this ^AE 2 3 ^AI 3 branch line 1 branch line 2 ... branch line 10000 ^AE 3 4 ^AD 2 5 ^AE 2 ^AE 1 So if I checked out 1.1.1.1, the set is 1, 3, I walk the weave and I'll print anything inserted by either of those, delete anything deleted by those, skip anything inserted by anything not in the set, skip any deletes by anything not in the set. The delta table looks like this, notice I've added an author column: rev me parent include exclude author 1.1.1.1 3 1 lm 1.2 2 1 lm 1.1 1 0 lm If you followed all that, you can see how SCCS can merge by reference. Lets say Clem decides to merge my branch onto the trunk. The delta table will get a new entry: rev me parent include exclude author 1.3 4 2 3 clem 1.1.1.1 3 1 lm 1.2 2 1 lm 1.1 1 0 lm The weave DOES NOT CHANGE. That's the pass by reference. You do the 3 way merge, it will find the lines "3" and "5" as anchor points in both versions, so it is a simple insert with no new data added to the weave. Here's some magic that *everyone* else gets wrong when they pass by value: In a system that passes by value (copies) the data, the merge done by clem would have an annotated listing like so: lm 1 lm 2 lm 3 clem branch line 1 clem branch line 2 clem ... clem branch line 10000 lm 4 lm 5 Since it copied the data, it looks like Clem wrote it but he didn't, he just automerged it. In SCCS/BitKeeper it would look like: lm 1 lm 2 lm 3 lm branch line 1 lm branch line 2 lm ... lm branch line 10000 lm 4 lm 5 which is correct, all of those lines were authored by one person. The only time the merger should show up as an author is if there was a conflict, however the merger resolved that conflict is new work and should be authored by the merger. What BitKeeper did, that was a profound step forward, was make the idea of a repository a formal thing and introduced the concept of changesets that keeps track of all this stuff at the repository level. So it does all this stuff at the file level but you don't have to do that low level work. You could think of SCCS as assembly and BitKeeper as more like C, it upleveled things to the point that humans can manage the repository easily. Whew. That's a butt load of info. Perhaps better for COFF? Any questions? It should be obvious that I *love* SCCS, it's a dramatically better file format than a patch based one, you can get *any* version of the file in constant time, authorship can be preserved across versions, it's pretty brilliant and I consider myself blessed to be posting this in response to SCCS's creator. Hats off to Marc. And big boo, hiss, to the RCS guy, who got a PhD for RCS (give me a break) and did the world a huge disservice by bad mouthing SCCS so he could promote RCS. --lm From mrochkind at gmail.com Sat Dec 14 04:32:57 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Fri, 13 Dec 2024 11:32:57 -0700 Subject: [TUHS] SCCS roach motel In-Reply-To: <20241213180649.GW11590@mcvoy.com> References: <20241213180649.GW11590@mcvoy.com> Message-ID: Larry, thanks for this. I had read some things you've written about the weave before, but not with this level of detail. Sounds weird, but I didn't really appreciate the implications of the weave even though I'm the guy who thought it up. I did understand the importance of not copying data if you can reference it, which is a principle of database design (normal forms, etc). In my paper, I can add a little more about the weave and its advantages. Aside from this TUHS post, is there something I can put in the References that people can find? Question: Is this right, that TeamWare was literally layered on top of AT&T SCCS, but BitKeeper was layered on your implementation of SCCS? Or, was it more complicated than that? Was your implementation of SCCS ever released by itself? Marc On Fri, Dec 13, 2024 at 11:06 AM Larry McVoy wrote: > On Fri, Dec 13, 2024 at 09:52:28AM -0700, Marc Rochkind wrote: > > IEEE Transactions on Software Engineering has asked me to write a > > retrospective on the influence of SCCS over the last 50 years, as my SCCS > > paper was published in 1975. They consider it one of the most influential > > papers from TSE's first decade. > > > > There's a funny quote from Ken Thompson that circulates from > time-to-time: > > > > "SCCS, the source motel! Programs check in and never check out!" > > > > But nobody seems to know what it means exactly. As part of my research, I > > asked Ken what the quote meant, sunce I wanted to include it. He > explained > > that it refers to SCCS storing binary data in its repository file, > > preventing UNIX text tools from operating on the file. > > > > Of course, this is only one of SCCS's many weaknesses. If you have > anything > > funny about any of the others, post it here. I already have all the > boring > > usual stuff (e.g., long-term locks, file-oriented, no merging). > > Warning, I know more about SCCS than the average person, I've > reimplemented it from scratch and then built BitKeeper on top of an > extended SCCS file format. So lots of info coming. Rick Smith and > Wayne Scott know as much as I do, Rick knows more, he joined me and > promptly started fixing my SCCS implementation. So far as I know, > there is not a more knowledgable person that Rick when it comes to > weave file formats. > > SCCS's strength is the weave format. It's largely not understood, even > by other people working in source management. Here's the benefit of > that weave (if people use it, which, other than BitKeeper, they don't, > looking at you, Clearcase, you had a weave and didn't use it): SCCS can > pass merge data by reference, everyone else copies the data. > > SCCS is a set based system. Each node has a revision number, like 1.5, > but because SCCS, unlike RCS, limited the revisions to at most 4 fields, > like 1.5.1.1, it is _impossible_ to build the history graph from the > revisions, you can in simple graphs but as soon as you branch from a > branch, all bets are off. > > The graph is built from what BitKeeper called serial numbers. Each node > in the graph has at least 2 serials, one that names that node in the > graph, and one that is the parent. > > So if I have a file with 5 revisions in straight line history, the > internal stuff will look something like > > rev me parent > 1.5 5 4 > 1.4 4 3 > 1.3 3 2 > 1.2 2 1 > 1.1 1 0 > > So what's the set? Pretty simple for straight line history, you walk > the history from the rev that you want, adding the "me" serial and > going to the parent, repeat until the parent is 0. > > Suppose you branch from rev 1.3. > > rev me parent > 1.3.1.1 6 3 > 1.5 5 4 > 1.4 4 3 > 1.3 3 2 > ... > > See that 1.3.1.1 is me: 6 and parent: 3. So if I were building the set > for 1.3.1.1, it becomes 6, then go to parent 3, 2, 1, skipping over 5 > and 4. If you understand that, you are starting to understand the set > and how it is constructed. > > Did you know you can have an argument in the revision history without > adding anything to the data part? SCCS has the ability to include > and/or exclude serials as part of a delta. Lets say Marc looked at > my 1.5 and thought it was garbage. He can exclude it from the > set like so: > > rev me parent include exclude > 1.6 7 5 0 5 > 1.3.1.1 6 3 > 1.5 5 4 > 1.4 4 3 > 1.3 3 2 > ... > > That doesn't change the data part of the file AT ALL, it's just saying > Marc doesn't want anyone to see the 1.5 changes. > > To understand that, you need to know how SCCS checks out a file. And > you need to know how the data is stored. Which is in a weave. RCS, > and pretty much everything that followed it, doesn't use a weave at > all. RCS stores the most recent version of the file as a complete > copy of the checked out file. Then each delta working backwards up > the trunk is a patch, what diff produces. > > Think about what that means for working on a branch. You have to start > with the most recent version of the file, apply backward patches to go > to earlier versions all the way back to the branch point, then apply > forward patches to work your way to tip of the branch. Ask Dave Miller > how pleasant it is to work on gcc on a branch. It's crazy slow and > painful. > > So how does SCCS do it? Lets say the first version of a file is > > 1 > 2 > 3 > 4 > 5 > > The data portion of the history file will look like: > > ^AI 1 > 1 > 2 > 3 > 4 > 5 > ^AE 1 > > SCCS used ^A at the beginning of a line to mean "this is metadata for > SCCS". ^AI is an insert, ^AD is a delete, and insert/delete are paired > with a ^AE which means end. The number after is the serial. So that > weave says "If serial 1 is in your set, everything after ^AI 1 is part > of that set until you hit the matching ^AE 1. > > Lets say the 2nd version is > > 1 > 2 > serial 2 added this > 3 > 4 > > Notice that serial 2 deleted what was line 5. > > ^AI 1 > 1 > 2 > ^AI 2 > serial 2 added this > ^AE 2 > 3 > 4 > ^AD 2 > 5 > ^AE 2 > ^AE 1 > > So now we can start to see how you walk the weave. If I'm trying to > check out 1.1 aka serial 1, I build a set that has only '1' in the set. > I hit the ^AI 1 see that I have 1 in my set, so I'm now in print mode, > which means print each data line. I hit ^AI 2, that's not in my set, > so I'm now in skip mode. And I skip the stuff inserted by serial 2. > I see the ^AE 2 and I revert back to print mode. I get to ^AD 2, > 2 is NOT in my set, so I stay in print mode. Etc. > > Let's make a branch, 1.1.1.1, with lots of data. > > 1 > 2 > 3 > branch line 1 > branch line 2 > ... > branch line 10000 > 4 > 5 > > ^AI 1 > 1 > 2 > ^AI 2 > serial 2 added this > ^AE 2 > 3 > ^AI 3 > branch line 1 > branch line 2 > ... > branch line 10000 > ^AE 3 > 4 > ^AD 2 > 5 > ^AE 2 > ^AE 1 > > So if I checked out 1.1.1.1, the set is 1, 3, I walk the weave and I'll > print anything inserted by either of those, delete anything deleted > by those, skip anything inserted by anything not in the set, skip any > deletes by anything not in the set. > > The delta table looks like this, notice I've added an author column: > > rev me parent include exclude author > 1.1.1.1 3 1 lm > 1.2 2 1 lm > 1.1 1 0 lm > > If you followed all that, you can see how SCCS can merge by reference. > Lets say Clem decides to merge my branch onto the trunk. The delta table > will get a new entry: > > rev me parent include exclude author > 1.3 4 2 3 clem > 1.1.1.1 3 1 lm > 1.2 2 1 lm > 1.1 1 0 lm > > The weave DOES NOT CHANGE. That's the pass by reference. You do the 3 way > merge, it will find the lines "3" and "5" as anchor points in both > versions, > so it is a simple insert with no new data added to the weave. > > Here's some magic that *everyone* else gets wrong when they pass by value: > In a system that passes by value (copies) the data, the merge done by clem > would have an annotated listing like so: > > lm 1 > lm 2 > lm 3 > clem branch line 1 > clem branch line 2 > clem ... > clem branch line 10000 > lm 4 > lm 5 > > Since it copied the data, it looks like Clem wrote it but he didn't, he > just automerged it. In SCCS/BitKeeper it would look like: > > lm 1 > lm 2 > lm 3 > lm branch line 1 > lm branch line 2 > lm ... > lm branch line 10000 > lm 4 > lm 5 > > which is correct, all of those lines were authored by one person. The only > time the merger should show up as an author is if there was a conflict, > however the merger resolved that conflict is new work and should be > authored by the merger. > > What BitKeeper did, that was a profound step forward, was make the idea > of a repository a formal thing and introduced the concept of changesets > that keeps track of all this stuff at the repository level. So it does > all this stuff at the file level but you don't have to do that low level > work. You could think of SCCS as assembly and BitKeeper as more like C, > it upleveled things to the point that humans can manage the repository > easily. > > Whew. That's a butt load of info. Perhaps better for COFF? Any > questions? It should be obvious that I *love* SCCS, it's a dramatically > better file format than a patch based one, you can get *any* version of > the file in constant time, authorship can be preserved across versions, > it's pretty brilliant and I consider myself blessed to be posting this > in response to SCCS's creator. Hats off to Marc. And big boo, hiss, > to the RCS guy, who got a PhD for RCS (give me a break) and did the > world a huge disservice by bad mouthing SCCS so he could promote RCS. > > --lm > -- *My new email address is mrochkind at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrochkind at gmail.com Sat Dec 14 04:39:03 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Fri, 13 Dec 2024 11:39:03 -0700 Subject: [TUHS] SCCS roach motel In-Reply-To: References: <20241213180649.GW11590@mcvoy.com> Message-ID: Larry, I found this: https://www.bitkeeper.org/src-notes/SCCSWEAVE.html A good reference? Marc On Fri, Dec 13, 2024 at 11:32 AM Marc Rochkind wrote: > Larry, thanks for this. I had read some things you've written about the > weave before, but not with this level of detail. Sounds weird, but I didn't > really appreciate the implications of the weave even though I'm the guy who > thought it up. I did understand the importance of not copying data if you > can reference it, which is a principle of database design (normal forms, > etc). > > In my paper, I can add a little more about the weave and its advantages. > Aside from this TUHS post, is there something I can put in the References > that people can find? > > Question: Is this right, that TeamWare was literally layered on top of > AT&T SCCS, but BitKeeper was layered on your implementation of SCCS? Or, > was it more complicated than that? > > Was your implementation of SCCS ever released by itself? > > Marc > > On Fri, Dec 13, 2024 at 11:06 AM Larry McVoy wrote: > >> On Fri, Dec 13, 2024 at 09:52:28AM -0700, Marc Rochkind wrote: >> > IEEE Transactions on Software Engineering has asked me to write a >> > retrospective on the influence of SCCS over the last 50 years, as my >> SCCS >> > paper was published in 1975. They consider it one of the most >> influential >> > papers from TSE's first decade. >> > >> > There's a funny quote from Ken Thompson that circulates from >> time-to-time: >> > >> > "SCCS, the source motel! Programs check in and never check out!" >> > >> > But nobody seems to know what it means exactly. As part of my research, >> I >> > asked Ken what the quote meant, sunce I wanted to include it. He >> explained >> > that it refers to SCCS storing binary data in its repository file, >> > preventing UNIX text tools from operating on the file. >> > >> > Of course, this is only one of SCCS's many weaknesses. If you have >> anything >> > funny about any of the others, post it here. I already have all the >> boring >> > usual stuff (e.g., long-term locks, file-oriented, no merging). >> >> Warning, I know more about SCCS than the average person, I've >> reimplemented it from scratch and then built BitKeeper on top of an >> extended SCCS file format. So lots of info coming. Rick Smith and >> Wayne Scott know as much as I do, Rick knows more, he joined me and >> promptly started fixing my SCCS implementation. So far as I know, >> there is not a more knowledgable person that Rick when it comes to >> weave file formats. >> >> SCCS's strength is the weave format. It's largely not understood, even >> by other people working in source management. Here's the benefit of >> that weave (if people use it, which, other than BitKeeper, they don't, >> looking at you, Clearcase, you had a weave and didn't use it): SCCS can >> pass merge data by reference, everyone else copies the data. >> >> SCCS is a set based system. Each node has a revision number, like 1.5, >> but because SCCS, unlike RCS, limited the revisions to at most 4 fields, >> like 1.5.1.1, it is _impossible_ to build the history graph from the >> revisions, you can in simple graphs but as soon as you branch from a >> branch, all bets are off. >> >> The graph is built from what BitKeeper called serial numbers. Each node >> in the graph has at least 2 serials, one that names that node in the >> graph, and one that is the parent. >> >> So if I have a file with 5 revisions in straight line history, the >> internal stuff will look something like >> >> rev me parent >> 1.5 5 4 >> 1.4 4 3 >> 1.3 3 2 >> 1.2 2 1 >> 1.1 1 0 >> >> So what's the set? Pretty simple for straight line history, you walk >> the history from the rev that you want, adding the "me" serial and >> going to the parent, repeat until the parent is 0. >> >> Suppose you branch from rev 1.3. >> >> rev me parent >> 1.3.1.1 6 3 >> 1.5 5 4 >> 1.4 4 3 >> 1.3 3 2 >> ... >> >> See that 1.3.1.1 is me: 6 and parent: 3. So if I were building the set >> for 1.3.1.1, it becomes 6, then go to parent 3, 2, 1, skipping over 5 >> and 4. If you understand that, you are starting to understand the set >> and how it is constructed. >> >> Did you know you can have an argument in the revision history without >> adding anything to the data part? SCCS has the ability to include >> and/or exclude serials as part of a delta. Lets say Marc looked at >> my 1.5 and thought it was garbage. He can exclude it from the >> set like so: >> >> rev me parent include exclude >> 1.6 7 5 0 5 >> 1.3.1.1 6 3 >> 1.5 5 4 >> 1.4 4 3 >> 1.3 3 2 >> ... >> >> That doesn't change the data part of the file AT ALL, it's just saying >> Marc doesn't want anyone to see the 1.5 changes. >> >> To understand that, you need to know how SCCS checks out a file. And >> you need to know how the data is stored. Which is in a weave. RCS, >> and pretty much everything that followed it, doesn't use a weave at >> all. RCS stores the most recent version of the file as a complete >> copy of the checked out file. Then each delta working backwards up >> the trunk is a patch, what diff produces. >> >> Think about what that means for working on a branch. You have to start >> with the most recent version of the file, apply backward patches to go >> to earlier versions all the way back to the branch point, then apply >> forward patches to work your way to tip of the branch. Ask Dave Miller >> how pleasant it is to work on gcc on a branch. It's crazy slow and >> painful. >> >> So how does SCCS do it? Lets say the first version of a file is >> >> 1 >> 2 >> 3 >> 4 >> 5 >> >> The data portion of the history file will look like: >> >> ^AI 1 >> 1 >> 2 >> 3 >> 4 >> 5 >> ^AE 1 >> >> SCCS used ^A at the beginning of a line to mean "this is metadata for >> SCCS". ^AI is an insert, ^AD is a delete, and insert/delete are paired >> with a ^AE which means end. The number after is the serial. So that >> weave says "If serial 1 is in your set, everything after ^AI 1 is part >> of that set until you hit the matching ^AE 1. >> >> Lets say the 2nd version is >> >> 1 >> 2 >> serial 2 added this >> 3 >> 4 >> >> Notice that serial 2 deleted what was line 5. >> >> ^AI 1 >> 1 >> 2 >> ^AI 2 >> serial 2 added this >> ^AE 2 >> 3 >> 4 >> ^AD 2 >> 5 >> ^AE 2 >> ^AE 1 >> >> So now we can start to see how you walk the weave. If I'm trying to >> check out 1.1 aka serial 1, I build a set that has only '1' in the set. >> I hit the ^AI 1 see that I have 1 in my set, so I'm now in print mode, >> which means print each data line. I hit ^AI 2, that's not in my set, >> so I'm now in skip mode. And I skip the stuff inserted by serial 2. >> I see the ^AE 2 and I revert back to print mode. I get to ^AD 2, >> 2 is NOT in my set, so I stay in print mode. Etc. >> >> Let's make a branch, 1.1.1.1, with lots of data. >> >> 1 >> 2 >> 3 >> branch line 1 >> branch line 2 >> ... >> branch line 10000 >> 4 >> 5 >> >> ^AI 1 >> 1 >> 2 >> ^AI 2 >> serial 2 added this >> ^AE 2 >> 3 >> ^AI 3 >> branch line 1 >> branch line 2 >> ... >> branch line 10000 >> ^AE 3 >> 4 >> ^AD 2 >> 5 >> ^AE 2 >> ^AE 1 >> >> So if I checked out 1.1.1.1, the set is 1, 3, I walk the weave and I'll >> print anything inserted by either of those, delete anything deleted >> by those, skip anything inserted by anything not in the set, skip any >> deletes by anything not in the set. >> >> The delta table looks like this, notice I've added an author column: >> >> rev me parent include exclude author >> 1.1.1.1 3 1 lm >> 1.2 2 1 lm >> 1.1 1 0 lm >> >> If you followed all that, you can see how SCCS can merge by reference. >> Lets say Clem decides to merge my branch onto the trunk. The delta table >> will get a new entry: >> >> rev me parent include exclude author >> 1.3 4 2 3 clem >> 1.1.1.1 3 1 lm >> 1.2 2 1 lm >> 1.1 1 0 lm >> >> The weave DOES NOT CHANGE. That's the pass by reference. You do the 3 >> way >> merge, it will find the lines "3" and "5" as anchor points in both >> versions, >> so it is a simple insert with no new data added to the weave. >> >> Here's some magic that *everyone* else gets wrong when they pass by value: >> In a system that passes by value (copies) the data, the merge done by clem >> would have an annotated listing like so: >> >> lm 1 >> lm 2 >> lm 3 >> clem branch line 1 >> clem branch line 2 >> clem ... >> clem branch line 10000 >> lm 4 >> lm 5 >> >> Since it copied the data, it looks like Clem wrote it but he didn't, he >> just automerged it. In SCCS/BitKeeper it would look like: >> >> lm 1 >> lm 2 >> lm 3 >> lm branch line 1 >> lm branch line 2 >> lm ... >> lm branch line 10000 >> lm 4 >> lm 5 >> >> which is correct, all of those lines were authored by one person. The >> only >> time the merger should show up as an author is if there was a conflict, >> however the merger resolved that conflict is new work and should be >> authored by the merger. >> >> What BitKeeper did, that was a profound step forward, was make the idea >> of a repository a formal thing and introduced the concept of changesets >> that keeps track of all this stuff at the repository level. So it does >> all this stuff at the file level but you don't have to do that low level >> work. You could think of SCCS as assembly and BitKeeper as more like C, >> it upleveled things to the point that humans can manage the repository >> easily. >> >> Whew. That's a butt load of info. Perhaps better for COFF? Any >> questions? It should be obvious that I *love* SCCS, it's a dramatically >> better file format than a patch based one, you can get *any* version of >> the file in constant time, authorship can be preserved across versions, >> it's pretty brilliant and I consider myself blessed to be posting this >> in response to SCCS's creator. Hats off to Marc. And big boo, hiss, >> to the RCS guy, who got a PhD for RCS (give me a break) and did the >> world a huge disservice by bad mouthing SCCS so he could promote RCS. >> >> --lm >> > > > -- > *My new email address is mrochkind at gmail.com * > -- *My new email address is mrochkind at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Sat Dec 14 04:49:43 2024 From: lm at mcvoy.com (Larry McVoy) Date: Fri, 13 Dec 2024 10:49:43 -0800 Subject: [TUHS] SCCS roach motel In-Reply-To: References: <20241213180649.GW11590@mcvoy.com> Message-ID: <20241213184943.GX11590@mcvoy.com> Yes, but note that when you see "^AcK" for varying values of "K", those are BitKeeper extensions to the file format. We figured out that SCCS treats "^Ac " the same whether there is a space or a character there. It was a way for us to add stuff to the file format where it would look weird when you printed the comments, but SCCS could, at least, read the BK file format. All of src-notes/ was true at some point but has almost certainly rotted. We wrote those as a way to get developers started in the right direction. The source code is the real reference. On Fri, Dec 13, 2024 at 11:39:03AM -0700, Marc Rochkind wrote: > Larry, I found this: > > https://www.bitkeeper.org/src-notes/SCCSWEAVE.html > > A good reference? > > Marc > > On Fri, Dec 13, 2024 at 11:32???AM Marc Rochkind wrote: > > > Larry, thanks for this. I had read some things you've written about the > > weave before, but not with this level of detail. Sounds weird, but I didn't > > really appreciate the implications of the weave even though I'm the guy who > > thought it up. I did understand the importance of not copying data if you > > can reference it, which is a principle of database design (normal forms, > > etc). > > > > In my paper, I can add a little more about the weave and its advantages. > > Aside from this TUHS post, is there something I can put in the References > > that people can find? > > > > Question: Is this right, that TeamWare was literally layered on top of > > AT&T SCCS, but BitKeeper was layered on your implementation of SCCS? Or, > > was it more complicated than that? > > > > Was your implementation of SCCS ever released by itself? > > > > Marc > > > > On Fri, Dec 13, 2024 at 11:06???AM Larry McVoy wrote: > > > >> On Fri, Dec 13, 2024 at 09:52:28AM -0700, Marc Rochkind wrote: > >> > IEEE Transactions on Software Engineering has asked me to write a > >> > retrospective on the influence of SCCS over the last 50 years, as my > >> SCCS > >> > paper was published in 1975. They consider it one of the most > >> influential > >> > papers from TSE's first decade. > >> > > >> > There's a funny quote from Ken Thompson that circulates from > >> time-to-time: > >> > > >> > "SCCS, the source motel! Programs check in and never check out!" > >> > > >> > But nobody seems to know what it means exactly. As part of my research, > >> I > >> > asked Ken what the quote meant, sunce I wanted to include it. He > >> explained > >> > that it refers to SCCS storing binary data in its repository file, > >> > preventing UNIX text tools from operating on the file. > >> > > >> > Of course, this is only one of SCCS's many weaknesses. If you have > >> anything > >> > funny about any of the others, post it here. I already have all the > >> boring > >> > usual stuff (e.g., long-term locks, file-oriented, no merging). > >> > >> Warning, I know more about SCCS than the average person, I've > >> reimplemented it from scratch and then built BitKeeper on top of an > >> extended SCCS file format. So lots of info coming. Rick Smith and > >> Wayne Scott know as much as I do, Rick knows more, he joined me and > >> promptly started fixing my SCCS implementation. So far as I know, > >> there is not a more knowledgable person that Rick when it comes to > >> weave file formats. > >> > >> SCCS's strength is the weave format. It's largely not understood, even > >> by other people working in source management. Here's the benefit of > >> that weave (if people use it, which, other than BitKeeper, they don't, > >> looking at you, Clearcase, you had a weave and didn't use it): SCCS can > >> pass merge data by reference, everyone else copies the data. > >> > >> SCCS is a set based system. Each node has a revision number, like 1.5, > >> but because SCCS, unlike RCS, limited the revisions to at most 4 fields, > >> like 1.5.1.1, it is _impossible_ to build the history graph from the > >> revisions, you can in simple graphs but as soon as you branch from a > >> branch, all bets are off. > >> > >> The graph is built from what BitKeeper called serial numbers. Each node > >> in the graph has at least 2 serials, one that names that node in the > >> graph, and one that is the parent. > >> > >> So if I have a file with 5 revisions in straight line history, the > >> internal stuff will look something like > >> > >> rev me parent > >> 1.5 5 4 > >> 1.4 4 3 > >> 1.3 3 2 > >> 1.2 2 1 > >> 1.1 1 0 > >> > >> So what's the set? Pretty simple for straight line history, you walk > >> the history from the rev that you want, adding the "me" serial and > >> going to the parent, repeat until the parent is 0. > >> > >> Suppose you branch from rev 1.3. > >> > >> rev me parent > >> 1.3.1.1 6 3 > >> 1.5 5 4 > >> 1.4 4 3 > >> 1.3 3 2 > >> ... > >> > >> See that 1.3.1.1 is me: 6 and parent: 3. So if I were building the set > >> for 1.3.1.1, it becomes 6, then go to parent 3, 2, 1, skipping over 5 > >> and 4. If you understand that, you are starting to understand the set > >> and how it is constructed. > >> > >> Did you know you can have an argument in the revision history without > >> adding anything to the data part? SCCS has the ability to include > >> and/or exclude serials as part of a delta. Lets say Marc looked at > >> my 1.5 and thought it was garbage. He can exclude it from the > >> set like so: > >> > >> rev me parent include exclude > >> 1.6 7 5 0 5 > >> 1.3.1.1 6 3 > >> 1.5 5 4 > >> 1.4 4 3 > >> 1.3 3 2 > >> ... > >> > >> That doesn't change the data part of the file AT ALL, it's just saying > >> Marc doesn't want anyone to see the 1.5 changes. > >> > >> To understand that, you need to know how SCCS checks out a file. And > >> you need to know how the data is stored. Which is in a weave. RCS, > >> and pretty much everything that followed it, doesn't use a weave at > >> all. RCS stores the most recent version of the file as a complete > >> copy of the checked out file. Then each delta working backwards up > >> the trunk is a patch, what diff produces. > >> > >> Think about what that means for working on a branch. You have to start > >> with the most recent version of the file, apply backward patches to go > >> to earlier versions all the way back to the branch point, then apply > >> forward patches to work your way to tip of the branch. Ask Dave Miller > >> how pleasant it is to work on gcc on a branch. It's crazy slow and > >> painful. > >> > >> So how does SCCS do it? Lets say the first version of a file is > >> > >> 1 > >> 2 > >> 3 > >> 4 > >> 5 > >> > >> The data portion of the history file will look like: > >> > >> ^AI 1 > >> 1 > >> 2 > >> 3 > >> 4 > >> 5 > >> ^AE 1 > >> > >> SCCS used ^A at the beginning of a line to mean "this is metadata for > >> SCCS". ^AI is an insert, ^AD is a delete, and insert/delete are paired > >> with a ^AE which means end. The number after is the serial. So that > >> weave says "If serial 1 is in your set, everything after ^AI 1 is part > >> of that set until you hit the matching ^AE 1. > >> > >> Lets say the 2nd version is > >> > >> 1 > >> 2 > >> serial 2 added this > >> 3 > >> 4 > >> > >> Notice that serial 2 deleted what was line 5. > >> > >> ^AI 1 > >> 1 > >> 2 > >> ^AI 2 > >> serial 2 added this > >> ^AE 2 > >> 3 > >> 4 > >> ^AD 2 > >> 5 > >> ^AE 2 > >> ^AE 1 > >> > >> So now we can start to see how you walk the weave. If I'm trying to > >> check out 1.1 aka serial 1, I build a set that has only '1' in the set. > >> I hit the ^AI 1 see that I have 1 in my set, so I'm now in print mode, > >> which means print each data line. I hit ^AI 2, that's not in my set, > >> so I'm now in skip mode. And I skip the stuff inserted by serial 2. > >> I see the ^AE 2 and I revert back to print mode. I get to ^AD 2, > >> 2 is NOT in my set, so I stay in print mode. Etc. > >> > >> Let's make a branch, 1.1.1.1, with lots of data. > >> > >> 1 > >> 2 > >> 3 > >> branch line 1 > >> branch line 2 > >> ... > >> branch line 10000 > >> 4 > >> 5 > >> > >> ^AI 1 > >> 1 > >> 2 > >> ^AI 2 > >> serial 2 added this > >> ^AE 2 > >> 3 > >> ^AI 3 > >> branch line 1 > >> branch line 2 > >> ... > >> branch line 10000 > >> ^AE 3 > >> 4 > >> ^AD 2 > >> 5 > >> ^AE 2 > >> ^AE 1 > >> > >> So if I checked out 1.1.1.1, the set is 1, 3, I walk the weave and I'll > >> print anything inserted by either of those, delete anything deleted > >> by those, skip anything inserted by anything not in the set, skip any > >> deletes by anything not in the set. > >> > >> The delta table looks like this, notice I've added an author column: > >> > >> rev me parent include exclude author > >> 1.1.1.1 3 1 lm > >> 1.2 2 1 lm > >> 1.1 1 0 lm > >> > >> If you followed all that, you can see how SCCS can merge by reference. > >> Lets say Clem decides to merge my branch onto the trunk. The delta table > >> will get a new entry: > >> > >> rev me parent include exclude author > >> 1.3 4 2 3 clem > >> 1.1.1.1 3 1 lm > >> 1.2 2 1 lm > >> 1.1 1 0 lm > >> > >> The weave DOES NOT CHANGE. That's the pass by reference. You do the 3 > >> way > >> merge, it will find the lines "3" and "5" as anchor points in both > >> versions, > >> so it is a simple insert with no new data added to the weave. > >> > >> Here's some magic that *everyone* else gets wrong when they pass by value: > >> In a system that passes by value (copies) the data, the merge done by clem > >> would have an annotated listing like so: > >> > >> lm 1 > >> lm 2 > >> lm 3 > >> clem branch line 1 > >> clem branch line 2 > >> clem ... > >> clem branch line 10000 > >> lm 4 > >> lm 5 > >> > >> Since it copied the data, it looks like Clem wrote it but he didn't, he > >> just automerged it. In SCCS/BitKeeper it would look like: > >> > >> lm 1 > >> lm 2 > >> lm 3 > >> lm branch line 1 > >> lm branch line 2 > >> lm ... > >> lm branch line 10000 > >> lm 4 > >> lm 5 > >> > >> which is correct, all of those lines were authored by one person. The > >> only > >> time the merger should show up as an author is if there was a conflict, > >> however the merger resolved that conflict is new work and should be > >> authored by the merger. > >> > >> What BitKeeper did, that was a profound step forward, was make the idea > >> of a repository a formal thing and introduced the concept of changesets > >> that keeps track of all this stuff at the repository level. So it does > >> all this stuff at the file level but you don't have to do that low level > >> work. You could think of SCCS as assembly and BitKeeper as more like C, > >> it upleveled things to the point that humans can manage the repository > >> easily. > >> > >> Whew. That's a butt load of info. Perhaps better for COFF? Any > >> questions? It should be obvious that I *love* SCCS, it's a dramatically > >> better file format than a patch based one, you can get *any* version of > >> the file in constant time, authorship can be preserved across versions, > >> it's pretty brilliant and I consider myself blessed to be posting this > >> in response to SCCS's creator. Hats off to Marc. And big boo, hiss, > >> to the RCS guy, who got a PhD for RCS (give me a break) and did the > >> world a huge disservice by bad mouthing SCCS so he could promote RCS. > >> > >> --lm > >> > > > > > > -- > > *My new email address is mrochkind at gmail.com * > > > > > -- > *My new email address is mrochkind at gmail.com * -- --- Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From lm at mcvoy.com Sat Dec 14 04:55:34 2024 From: lm at mcvoy.com (Larry McVoy) Date: Fri, 13 Dec 2024 10:55:34 -0800 Subject: [TUHS] SCCS roach motel In-Reply-To: References: <20241213180649.GW11590@mcvoy.com> Message-ID: <20241213185534.GY11590@mcvoy.com> On Fri, Dec 13, 2024 at 11:32:57AM -0700, Marc Rochkind wrote: > Larry, thanks for this. I had read some things you've written about the > weave before, but not with this level of detail. Sounds weird, but I didn't > really appreciate the implications of the weave even though I'm the guy who > thought it up. I did understand the importance of not copying data if you > can reference it, which is a principle of database design (normal forms, > etc). > > In my paper, I can add a little more about the weave and its advantages. > Aside from this TUHS post, is there something I can put in the References > that people can find? > > Question: Is this right, that TeamWare was literally layered on top of AT&T > SCCS, but BitKeeper was layered on your implementation of SCCS? Or, was it > more complicated than that? Yeah, I wrote TeamWare as well though the source management group took all my perl4 and rewrote it in C++ (and regretted it). The only C program, smoosh.c, which zippered together 2 SCCS files that had at least 1.1 in common, they checked in without preserving my authorship. Yes, douche move, but they hated me because I was a kernel engineer, doing well at that, and I wrote NSElite in my spare time. NSElite was what they took and turned into TeamWare. The sad thing, for them, is that if they had let me keep going I would have evolved NSElite into BitKeeper, I was already thinking of changesets back them. They weren't the brightest people, they could copy what I did and that was about it. > Was your implementation of SCCS ever released by itself? See if the wayback machine has BitSCCS somewhere. It was pretty early, before Rick showed up to fix my screwups. He did point out that my weave implementation was the only one written such that I could have N serial sets in my hand, and do one pass through the weave and get N different checked out files. I don't think we ever used that but if we did it would be in smerge.c. > Marc > > On Fri, Dec 13, 2024 at 11:06???AM Larry McVoy wrote: > > > On Fri, Dec 13, 2024 at 09:52:28AM -0700, Marc Rochkind wrote: > > > IEEE Transactions on Software Engineering has asked me to write a > > > retrospective on the influence of SCCS over the last 50 years, as my SCCS > > > paper was published in 1975. They consider it one of the most influential > > > papers from TSE's first decade. > > > > > > There's a funny quote from Ken Thompson that circulates from > > time-to-time: > > > > > > "SCCS, the source motel! Programs check in and never check out!" > > > > > > But nobody seems to know what it means exactly. As part of my research, I > > > asked Ken what the quote meant, sunce I wanted to include it. He > > explained > > > that it refers to SCCS storing binary data in its repository file, > > > preventing UNIX text tools from operating on the file. > > > > > > Of course, this is only one of SCCS's many weaknesses. If you have > > anything > > > funny about any of the others, post it here. I already have all the > > boring > > > usual stuff (e.g., long-term locks, file-oriented, no merging). > > > > Warning, I know more about SCCS than the average person, I've > > reimplemented it from scratch and then built BitKeeper on top of an > > extended SCCS file format. So lots of info coming. Rick Smith and > > Wayne Scott know as much as I do, Rick knows more, he joined me and > > promptly started fixing my SCCS implementation. So far as I know, > > there is not a more knowledgable person that Rick when it comes to > > weave file formats. > > > > SCCS's strength is the weave format. It's largely not understood, even > > by other people working in source management. Here's the benefit of > > that weave (if people use it, which, other than BitKeeper, they don't, > > looking at you, Clearcase, you had a weave and didn't use it): SCCS can > > pass merge data by reference, everyone else copies the data. > > > > SCCS is a set based system. Each node has a revision number, like 1.5, > > but because SCCS, unlike RCS, limited the revisions to at most 4 fields, > > like 1.5.1.1, it is _impossible_ to build the history graph from the > > revisions, you can in simple graphs but as soon as you branch from a > > branch, all bets are off. > > > > The graph is built from what BitKeeper called serial numbers. Each node > > in the graph has at least 2 serials, one that names that node in the > > graph, and one that is the parent. > > > > So if I have a file with 5 revisions in straight line history, the > > internal stuff will look something like > > > > rev me parent > > 1.5 5 4 > > 1.4 4 3 > > 1.3 3 2 > > 1.2 2 1 > > 1.1 1 0 > > > > So what's the set? Pretty simple for straight line history, you walk > > the history from the rev that you want, adding the "me" serial and > > going to the parent, repeat until the parent is 0. > > > > Suppose you branch from rev 1.3. > > > > rev me parent > > 1.3.1.1 6 3 > > 1.5 5 4 > > 1.4 4 3 > > 1.3 3 2 > > ... > > > > See that 1.3.1.1 is me: 6 and parent: 3. So if I were building the set > > for 1.3.1.1, it becomes 6, then go to parent 3, 2, 1, skipping over 5 > > and 4. If you understand that, you are starting to understand the set > > and how it is constructed. > > > > Did you know you can have an argument in the revision history without > > adding anything to the data part? SCCS has the ability to include > > and/or exclude serials as part of a delta. Lets say Marc looked at > > my 1.5 and thought it was garbage. He can exclude it from the > > set like so: > > > > rev me parent include exclude > > 1.6 7 5 0 5 > > 1.3.1.1 6 3 > > 1.5 5 4 > > 1.4 4 3 > > 1.3 3 2 > > ... > > > > That doesn't change the data part of the file AT ALL, it's just saying > > Marc doesn't want anyone to see the 1.5 changes. > > > > To understand that, you need to know how SCCS checks out a file. And > > you need to know how the data is stored. Which is in a weave. RCS, > > and pretty much everything that followed it, doesn't use a weave at > > all. RCS stores the most recent version of the file as a complete > > copy of the checked out file. Then each delta working backwards up > > the trunk is a patch, what diff produces. > > > > Think about what that means for working on a branch. You have to start > > with the most recent version of the file, apply backward patches to go > > to earlier versions all the way back to the branch point, then apply > > forward patches to work your way to tip of the branch. Ask Dave Miller > > how pleasant it is to work on gcc on a branch. It's crazy slow and > > painful. > > > > So how does SCCS do it? Lets say the first version of a file is > > > > 1 > > 2 > > 3 > > 4 > > 5 > > > > The data portion of the history file will look like: > > > > ^AI 1 > > 1 > > 2 > > 3 > > 4 > > 5 > > ^AE 1 > > > > SCCS used ^A at the beginning of a line to mean "this is metadata for > > SCCS". ^AI is an insert, ^AD is a delete, and insert/delete are paired > > with a ^AE which means end. The number after is the serial. So that > > weave says "If serial 1 is in your set, everything after ^AI 1 is part > > of that set until you hit the matching ^AE 1. > > > > Lets say the 2nd version is > > > > 1 > > 2 > > serial 2 added this > > 3 > > 4 > > > > Notice that serial 2 deleted what was line 5. > > > > ^AI 1 > > 1 > > 2 > > ^AI 2 > > serial 2 added this > > ^AE 2 > > 3 > > 4 > > ^AD 2 > > 5 > > ^AE 2 > > ^AE 1 > > > > So now we can start to see how you walk the weave. If I'm trying to > > check out 1.1 aka serial 1, I build a set that has only '1' in the set. > > I hit the ^AI 1 see that I have 1 in my set, so I'm now in print mode, > > which means print each data line. I hit ^AI 2, that's not in my set, > > so I'm now in skip mode. And I skip the stuff inserted by serial 2. > > I see the ^AE 2 and I revert back to print mode. I get to ^AD 2, > > 2 is NOT in my set, so I stay in print mode. Etc. > > > > Let's make a branch, 1.1.1.1, with lots of data. > > > > 1 > > 2 > > 3 > > branch line 1 > > branch line 2 > > ... > > branch line 10000 > > 4 > > 5 > > > > ^AI 1 > > 1 > > 2 > > ^AI 2 > > serial 2 added this > > ^AE 2 > > 3 > > ^AI 3 > > branch line 1 > > branch line 2 > > ... > > branch line 10000 > > ^AE 3 > > 4 > > ^AD 2 > > 5 > > ^AE 2 > > ^AE 1 > > > > So if I checked out 1.1.1.1, the set is 1, 3, I walk the weave and I'll > > print anything inserted by either of those, delete anything deleted > > by those, skip anything inserted by anything not in the set, skip any > > deletes by anything not in the set. > > > > The delta table looks like this, notice I've added an author column: > > > > rev me parent include exclude author > > 1.1.1.1 3 1 lm > > 1.2 2 1 lm > > 1.1 1 0 lm > > > > If you followed all that, you can see how SCCS can merge by reference. > > Lets say Clem decides to merge my branch onto the trunk. The delta table > > will get a new entry: > > > > rev me parent include exclude author > > 1.3 4 2 3 clem > > 1.1.1.1 3 1 lm > > 1.2 2 1 lm > > 1.1 1 0 lm > > > > The weave DOES NOT CHANGE. That's the pass by reference. You do the 3 way > > merge, it will find the lines "3" and "5" as anchor points in both > > versions, > > so it is a simple insert with no new data added to the weave. > > > > Here's some magic that *everyone* else gets wrong when they pass by value: > > In a system that passes by value (copies) the data, the merge done by clem > > would have an annotated listing like so: > > > > lm 1 > > lm 2 > > lm 3 > > clem branch line 1 > > clem branch line 2 > > clem ... > > clem branch line 10000 > > lm 4 > > lm 5 > > > > Since it copied the data, it looks like Clem wrote it but he didn't, he > > just automerged it. In SCCS/BitKeeper it would look like: > > > > lm 1 > > lm 2 > > lm 3 > > lm branch line 1 > > lm branch line 2 > > lm ... > > lm branch line 10000 > > lm 4 > > lm 5 > > > > which is correct, all of those lines were authored by one person. The only > > time the merger should show up as an author is if there was a conflict, > > however the merger resolved that conflict is new work and should be > > authored by the merger. > > > > What BitKeeper did, that was a profound step forward, was make the idea > > of a repository a formal thing and introduced the concept of changesets > > that keeps track of all this stuff at the repository level. So it does > > all this stuff at the file level but you don't have to do that low level > > work. You could think of SCCS as assembly and BitKeeper as more like C, > > it upleveled things to the point that humans can manage the repository > > easily. > > > > Whew. That's a butt load of info. Perhaps better for COFF? Any > > questions? It should be obvious that I *love* SCCS, it's a dramatically > > better file format than a patch based one, you can get *any* version of > > the file in constant time, authorship can be preserved across versions, > > it's pretty brilliant and I consider myself blessed to be posting this > > in response to SCCS's creator. Hats off to Marc. And big boo, hiss, > > to the RCS guy, who got a PhD for RCS (give me a break) and did the > > world a huge disservice by bad mouthing SCCS so he could promote RCS. > > > > --lm > > > > > -- > *My new email address is mrochkind at gmail.com * -- --- Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From henry.r.bent at gmail.com Sat Dec 14 05:55:58 2024 From: henry.r.bent at gmail.com (Henry Bent) Date: Fri, 13 Dec 2024 14:55:58 -0500 Subject: [TUHS] SCCS roach motel In-Reply-To: <20241213185534.GY11590@mcvoy.com> References: <20241213180649.GW11590@mcvoy.com> <20241213185534.GY11590@mcvoy.com> Message-ID: On Fri, 13 Dec 2024 at 14:03, Larry McVoy wrote: > On Fri, Dec 13, 2024 at 11:32:57AM -0700, Marc Rochkind wrote: > > > Was your implementation of SCCS ever released by itself? > > See if the wayback machine has BitSCCS somewhere. It was pretty early, > before Rick showed up to fix my screwups. He did point out that my > weave implementation was the only one written such that I could have > N serial sets in my hand, and do one pass through the weave and get > N different checked out files. I don't think we ever used that but > if we did it would be in smerge.c. > There are many preservations of http://www.bitmover.com/bitsccs/, but since the BitSCCS sources were distributed via FTP the wayback machine doesn't have the actual sources. I did a bit of searching and found ftp://linux.mathematik.tu-darmstadt.de/pub/linux/distributions/historic/jurix/source/compile/BitSCCS-0.5.2.tar.gz ; I don't know where that falls in the product's lifetime. -Henry -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Sat Dec 14 07:09:35 2024 From: crossd at gmail.com (Dan Cross) Date: Fri, 13 Dec 2024 16:09:35 -0500 Subject: [TUHS] SCCS roach motel In-Reply-To: References: Message-ID: On Fri, Dec 13, 2024 at 3:28 PM Marc Rochkind wrote: > On Fri, Dec 13, 2024 at 10:03 AM Mark Seiden wrote: >> (I know there is a special place in hell for those who explain a joke, but, you asked…) >> >> it’s just an allusion to the Black Flag Roach Motel product (still being produced) >> which has a trademark on the phrase “Roaches Check in… But they Don’t Check Out”. > > Yeah, I knew that much. My question to Ken was about what this was saying about SCCS. I had heard a story once, that based on what you wrote today, I now think is apocryphal and never actually happened. The story was that there was a bug in early SCCS where a source file had to be checked in twice before it could be (successfully) checked out. The bug was such that if one only checked it in once and tried to check it out, it would truncate the file. "Programmers using it just learned to check in twice." I never used SCCS extensively, and certainly never observed that behavior, so chucked it up to the bug having been long fixed. As I mentioned above, I doubt it was ever there to begin with. A fun story, though. - Dan C. From robpike at gmail.com Sat Dec 14 07:22:41 2024 From: robpike at gmail.com (Rob Pike) Date: Sat, 14 Dec 2024 08:22:41 +1100 Subject: [TUHS] SCCS roach motel In-Reply-To: References: Message-ID: According to the Unix room fortunes file, the actual quote is SCCS: the source-code motel -- your code checks in but it never checks out. Ken Thompson On Sat, Dec 14, 2024 at 3:52 AM Marc Rochkind wrote: > IEEE Transactions on Software Engineering has asked me to write a > retrospective on the influence of SCCS over the last 50 years, as my SCCS > paper was published in 1975. They consider it one of the most influential > papers from TSE's first decade. > > There's a funny quote from Ken Thompson that circulates from time-to-time: > > "SCCS, the source motel! Programs check in and never check out!" > > But nobody seems to know what it means exactly. As part of my research, I > asked Ken what the quote meant, sunce I wanted to include it. He explained > that it refers to SCCS storing binary data in its repository file, > preventing UNIX text tools from operating on the file. > > Of course, this is only one of SCCS's many weaknesses. If you have > anything funny about any of the others, post it here. I already have all > the boring usual stuff (e.g., long-term locks, file-oriented, no merging). > > Marc Rochkind > mrochkind.com > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robpike at gmail.com Sat Dec 14 07:27:01 2024 From: robpike at gmail.com (Rob Pike) Date: Sat, 14 Dec 2024 08:27:01 +1100 Subject: [TUHS] SCCS roach motel In-Reply-To: References: Message-ID: To answer your actual question, it is of course a riff on a TV ad for a cockroach trap in the 1980s. The sentiment of the quote, as I saw it (it's possible I was the one who added it to the fortunes file after ken saw the SCCS burble at the top of some file from USG and laughed), was primarily a reaction to the taint it added to the previously annotation-free top of the file. It was also a response to the march of corporate code management stepping into the research world, or perhaps the hacker world. It's a philosophical thing, a feeling, not an argument. It all seems so quaint now. -rob On Sat, Dec 14, 2024 at 8:22 AM Rob Pike wrote: > According to the Unix room fortunes file, the actual quote is > > SCCS: the source-code motel -- your code checks in but it never checks out. > Ken Thompson > > On Sat, Dec 14, 2024 at 3:52 AM Marc Rochkind wrote: > >> IEEE Transactions on Software Engineering has asked me to write a >> retrospective on the influence of SCCS over the last 50 years, as my SCCS >> paper was published in 1975. They consider it one of the most influential >> papers from TSE's first decade. >> >> There's a funny quote from Ken Thompson that circulates from time-to-time: >> >> "SCCS, the source motel! Programs check in and never check out!" >> >> But nobody seems to know what it means exactly. As part of my research, I >> asked Ken what the quote meant, sunce I wanted to include it. He explained >> that it refers to SCCS storing binary data in its repository file, >> preventing UNIX text tools from operating on the file. >> >> Of course, this is only one of SCCS's many weaknesses. If you have >> anything funny about any of the others, post it here. I already have all >> the boring usual stuff (e.g., long-term locks, file-oriented, no merging). >> >> Marc Rochkind >> mrochkind.com >> >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From aki at insinga.com Sat Dec 14 07:37:46 2024 From: aki at insinga.com (Aron Insinga) Date: Fri, 13 Dec 2024 16:37:46 -0500 Subject: [TUHS] SCCS roach motel In-Reply-To: References: Message-ID: <9777d115-a0bd-4b5a-9436-4a0670b79568@insinga.com> The product and its advertising tag line still exist: https://www.homedepot.com/p/Black-Flag-Roach-Motel-Insect-Glue-Traps-2-Count-HG-11020-1/204237338 https://images.thdstatic.com/productImages/e8dd1acb-7a49-40ce-8ae2-c582c8bda9a2/svn/brown-black-flag-insect-traps-hg-11020-1-64_1000.jpg Note the red rectangle-with-rounded-corners under the large yellow word 'Motel' on the front of the box. - Aron On 12/13/24 16:27, Rob Pike wrote: > To answer your actual question, it is of course a riff on a TV ad for > a cockroach trap in the 1980s. The sentiment of the quote, as I saw it > (it's possible I was the one who added it to the fortunes file after > ken saw the SCCS burble at the top of some file from USG and laughed), > was primarily a reaction to the taint it added to the previously > annotation-free top of the file. It was also a response to the march > of corporate code management stepping into the research world, or > perhaps the hacker world. It's a philosophical thing, a feeling, not > an argument. > > It all seems so quaint now. > > -rob > > > On Sat, Dec 14, 2024 at 8:22 AM Rob Pike wrote: > > According to the Unix room fortunes file, the actual quote is > > SCCS: the source-code motel -- your code checks in but it never > checks out. > Ken Thompson > > On Sat, Dec 14, 2024 at 3:52 AM Marc Rochkind > wrote: > > IEEE Transactions on Software Engineering has asked me to > write a retrospective on the influence of SCCS over the last > 50 years, as my SCCS paper was published in 1975. They > consider it one of the most influential papers from TSE's > first decade. > > There's a funny quote from Ken Thompson that circulates from > time-to-time: > > "SCCS, the source motel! Programs check in and never check out!" > > But nobody seems to know what it means exactly. As part of my > research, I asked Ken what the quote meant, sunce I wanted to > include it. He explained that it refers to SCCS storing binary > data in its repository file, preventing UNIX text tools from > operating on the file. > > Of course, this is only one of SCCS's many weaknesses. If you > have anything funny about any of the others, post it here. I > already have all the boring usual stuff (e.g., long-term > locks, file-oriented, no merging). > > Marc Rochkind > mrochkind.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aki at insinga.com Sat Dec 14 07:40:02 2024 From: aki at insinga.com (Aron Insinga) Date: Fri, 13 Dec 2024 16:40:02 -0500 Subject: [TUHS] SCCS roach motel In-Reply-To: <9777d115-a0bd-4b5a-9436-4a0670b79568@insinga.com> References: <9777d115-a0bd-4b5a-9436-4a0670b79568@insinga.com> Message-ID: p.s. I should have looked at more hits before sending that.  Here's the ad (or one of them): https://www.youtube.com/watch?v=jKhGHxO-woc - Aron On 12/13/24 16:37, Aron Insinga wrote: > The product and its advertising tag line still exist: > > https://www.homedepot.com/p/Black-Flag-Roach-Motel-Insect-Glue-Traps-2-Count-HG-11020-1/204237338 > > https://images.thdstatic.com/productImages/e8dd1acb-7a49-40ce-8ae2-c582c8bda9a2/svn/brown-black-flag-insect-traps-hg-11020-1-64_1000.jpg > > Note the red rectangle-with-rounded-corners under the large yellow > word 'Motel' on the front of the box. > > - Aron > > > On 12/13/24 16:27, Rob Pike wrote: >> To answer your actual question, it is of course a riff on a TV ad for >> a cockroach trap in the 1980s. The sentiment of the quote, as I saw >> it (it's possible I was the one who added it to the fortunes file >> after ken saw the SCCS burble at the top of some file from USG and >> laughed), was primarily a reaction to the taint it added to the >> previously annotation-free top of the file. It was also a response to >> the march of corporate code management stepping into the research >> world, or perhaps the hacker world. It's a philosophical thing, a >> feeling, not an argument. >> >> It all seems so quaint now. >> >> -rob >> >> >> On Sat, Dec 14, 2024 at 8:22 AM Rob Pike wrote: >> >> According to the Unix room fortunes file, the actual quote is >> >> SCCS: the source-code motel -- your code checks in but it never >> checks out. >> Ken Thompson >> >> On Sat, Dec 14, 2024 at 3:52 AM Marc Rochkind >> wrote: >> >> IEEE Transactions on Software Engineering has asked me to >> write a retrospective on the influence of SCCS over the last >> 50 years, as my SCCS paper was published in 1975. They >> consider it one of the most influential papers from TSE's >> first decade. >> >> There's a funny quote from Ken Thompson that circulates from >> time-to-time: >> >> "SCCS, the source motel! Programs check in and never check out!" >> >> But nobody seems to know what it means exactly. As part of my >> research, I asked Ken what the quote meant, sunce I wanted to >> include it. He explained that it refers to SCCS storing >> binary data in its repository file, preventing UNIX text >> tools from operating on the file. >> >> Of course, this is only one of SCCS's many weaknesses. If you >> have anything funny about any of the others, post it here. I >> already have all the boring usual stuff (e.g., long-term >> locks, file-oriented, no merging). >> >> Marc Rochkind >> mrochkind.com >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From clemc at ccc.com Sat Dec 14 07:46:58 2024 From: clemc at ccc.com (Clem Cole) Date: Fri, 13 Dec 2024 16:46:58 -0500 Subject: [TUHS] SCCS roach motel In-Reply-To: References: <20241213180649.GW11590@mcvoy.com> Message-ID: @Marc and Larry As a satisfied user of SCCS (and later Bitkeeper), it's still my preferred choice. To this day, I have looked up directions to do simple things in Git that were so natural for me in SCCS. I don't think it's the old dog syndrome, either. SCCS was hardly perfect, but it solved a problem very well. Eric's sccs(1) front end for it from UCB cleaned up a few of the rough edges and experience taught us a little about care and feeding. Truth is I still use for small projects. It's easier to set up and it just protected me against myself. As a side note, it also exposed/demonstrated my real dislike for NFS early on when we started to see ZERO filled blocks in SCCS files (stateless just sucks). So thank you both. I have no idea how many times you saved my team and me time and bailed us out. ᐧ On Fri, Dec 13, 2024 at 1:33 PM Marc Rochkind wrote: > Larry, thanks for this. I had read some things you've written about the > weave before, but not with this level of detail. Sounds weird, but I didn't > really appreciate the implications of the weave even though I'm the guy who > thought it up. I did understand the importance of not copying data if you > can reference it, which is a principle of database design (normal forms, > etc). > > In my paper, I can add a little more about the weave and its advantages. > Aside from this TUHS post, is there something I can put in the References > that people can find? > > Question: Is this right, that TeamWare was literally layered on top of > AT&T SCCS, but BitKeeper was layered on your implementation of SCCS? Or, > was it more complicated than that? > > Was your implementation of SCCS ever released by itself? > > Marc > > On Fri, Dec 13, 2024 at 11:06 AM Larry McVoy wrote: > >> On Fri, Dec 13, 2024 at 09:52:28AM -0700, Marc Rochkind wrote: >> > IEEE Transactions on Software Engineering has asked me to write a >> > retrospective on the influence of SCCS over the last 50 years, as my >> SCCS >> > paper was published in 1975. They consider it one of the most >> influential >> > papers from TSE's first decade. >> > >> > There's a funny quote from Ken Thompson that circulates from >> time-to-time: >> > >> > "SCCS, the source motel! Programs check in and never check out!" >> > >> > But nobody seems to know what it means exactly. As part of my research, >> I >> > asked Ken what the quote meant, sunce I wanted to include it. He >> explained >> > that it refers to SCCS storing binary data in its repository file, >> > preventing UNIX text tools from operating on the file. >> > >> > Of course, this is only one of SCCS's many weaknesses. If you have >> anything >> > funny about any of the others, post it here. I already have all the >> boring >> > usual stuff (e.g., long-term locks, file-oriented, no merging). >> >> Warning, I know more about SCCS than the average person, I've >> reimplemented it from scratch and then built BitKeeper on top of an >> extended SCCS file format. So lots of info coming. Rick Smith and >> Wayne Scott know as much as I do, Rick knows more, he joined me and >> promptly started fixing my SCCS implementation. So far as I know, >> there is not a more knowledgable person that Rick when it comes to >> weave file formats. >> >> SCCS's strength is the weave format. It's largely not understood, even >> by other people working in source management. Here's the benefit of >> that weave (if people use it, which, other than BitKeeper, they don't, >> looking at you, Clearcase, you had a weave and didn't use it): SCCS can >> pass merge data by reference, everyone else copies the data. >> >> SCCS is a set based system. Each node has a revision number, like 1.5, >> but because SCCS, unlike RCS, limited the revisions to at most 4 fields, >> like 1.5.1.1, it is _impossible_ to build the history graph from the >> revisions, you can in simple graphs but as soon as you branch from a >> branch, all bets are off. >> >> The graph is built from what BitKeeper called serial numbers. Each node >> in the graph has at least 2 serials, one that names that node in the >> graph, and one that is the parent. >> >> So if I have a file with 5 revisions in straight line history, the >> internal stuff will look something like >> >> rev me parent >> 1.5 5 4 >> 1.4 4 3 >> 1.3 3 2 >> 1.2 2 1 >> 1.1 1 0 >> >> So what's the set? Pretty simple for straight line history, you walk >> the history from the rev that you want, adding the "me" serial and >> going to the parent, repeat until the parent is 0. >> >> Suppose you branch from rev 1.3. >> >> rev me parent >> 1.3.1.1 6 3 >> 1.5 5 4 >> 1.4 4 3 >> 1.3 3 2 >> ... >> >> See that 1.3.1.1 is me: 6 and parent: 3. So if I were building the set >> for 1.3.1.1, it becomes 6, then go to parent 3, 2, 1, skipping over 5 >> and 4. If you understand that, you are starting to understand the set >> and how it is constructed. >> >> Did you know you can have an argument in the revision history without >> adding anything to the data part? SCCS has the ability to include >> and/or exclude serials as part of a delta. Lets say Marc looked at >> my 1.5 and thought it was garbage. He can exclude it from the >> set like so: >> >> rev me parent include exclude >> 1.6 7 5 0 5 >> 1.3.1.1 6 3 >> 1.5 5 4 >> 1.4 4 3 >> 1.3 3 2 >> ... >> >> That doesn't change the data part of the file AT ALL, it's just saying >> Marc doesn't want anyone to see the 1.5 changes. >> >> To understand that, you need to know how SCCS checks out a file. And >> you need to know how the data is stored. Which is in a weave. RCS, >> and pretty much everything that followed it, doesn't use a weave at >> all. RCS stores the most recent version of the file as a complete >> copy of the checked out file. Then each delta working backwards up >> the trunk is a patch, what diff produces. >> >> Think about what that means for working on a branch. You have to start >> with the most recent version of the file, apply backward patches to go >> to earlier versions all the way back to the branch point, then apply >> forward patches to work your way to tip of the branch. Ask Dave Miller >> how pleasant it is to work on gcc on a branch. It's crazy slow and >> painful. >> >> So how does SCCS do it? Lets say the first version of a file is >> >> 1 >> 2 >> 3 >> 4 >> 5 >> >> The data portion of the history file will look like: >> >> ^AI 1 >> 1 >> 2 >> 3 >> 4 >> 5 >> ^AE 1 >> >> SCCS used ^A at the beginning of a line to mean "this is metadata for >> SCCS". ^AI is an insert, ^AD is a delete, and insert/delete are paired >> with a ^AE which means end. The number after is the serial. So that >> weave says "If serial 1 is in your set, everything after ^AI 1 is part >> of that set until you hit the matching ^AE 1. >> >> Lets say the 2nd version is >> >> 1 >> 2 >> serial 2 added this >> 3 >> 4 >> >> Notice that serial 2 deleted what was line 5. >> >> ^AI 1 >> 1 >> 2 >> ^AI 2 >> serial 2 added this >> ^AE 2 >> 3 >> 4 >> ^AD 2 >> 5 >> ^AE 2 >> ^AE 1 >> >> So now we can start to see how you walk the weave. If I'm trying to >> check out 1.1 aka serial 1, I build a set that has only '1' in the set. >> I hit the ^AI 1 see that I have 1 in my set, so I'm now in print mode, >> which means print each data line. I hit ^AI 2, that's not in my set, >> so I'm now in skip mode. And I skip the stuff inserted by serial 2. >> I see the ^AE 2 and I revert back to print mode. I get to ^AD 2, >> 2 is NOT in my set, so I stay in print mode. Etc. >> >> Let's make a branch, 1.1.1.1, with lots of data. >> >> 1 >> 2 >> 3 >> branch line 1 >> branch line 2 >> ... >> branch line 10000 >> 4 >> 5 >> >> ^AI 1 >> 1 >> 2 >> ^AI 2 >> serial 2 added this >> ^AE 2 >> 3 >> ^AI 3 >> branch line 1 >> branch line 2 >> ... >> branch line 10000 >> ^AE 3 >> 4 >> ^AD 2 >> 5 >> ^AE 2 >> ^AE 1 >> >> So if I checked out 1.1.1.1, the set is 1, 3, I walk the weave and I'll >> print anything inserted by either of those, delete anything deleted >> by those, skip anything inserted by anything not in the set, skip any >> deletes by anything not in the set. >> >> The delta table looks like this, notice I've added an author column: >> >> rev me parent include exclude author >> 1.1.1.1 3 1 lm >> 1.2 2 1 lm >> 1.1 1 0 lm >> >> If you followed all that, you can see how SCCS can merge by reference. >> Lets say Clem decides to merge my branch onto the trunk. The delta table >> will get a new entry: >> >> rev me parent include exclude author >> 1.3 4 2 3 clem >> 1.1.1.1 3 1 lm >> 1.2 2 1 lm >> 1.1 1 0 lm >> >> The weave DOES NOT CHANGE. That's the pass by reference. You do the 3 >> way >> merge, it will find the lines "3" and "5" as anchor points in both >> versions, >> so it is a simple insert with no new data added to the weave. >> >> Here's some magic that *everyone* else gets wrong when they pass by value: >> In a system that passes by value (copies) the data, the merge done by clem >> would have an annotated listing like so: >> >> lm 1 >> lm 2 >> lm 3 >> clem branch line 1 >> clem branch line 2 >> clem ... >> clem branch line 10000 >> lm 4 >> lm 5 >> >> Since it copied the data, it looks like Clem wrote it but he didn't, he >> just automerged it. In SCCS/BitKeeper it would look like: >> >> lm 1 >> lm 2 >> lm 3 >> lm branch line 1 >> lm branch line 2 >> lm ... >> lm branch line 10000 >> lm 4 >> lm 5 >> >> which is correct, all of those lines were authored by one person. The >> only >> time the merger should show up as an author is if there was a conflict, >> however the merger resolved that conflict is new work and should be >> authored by the merger. >> >> What BitKeeper did, that was a profound step forward, was make the idea >> of a repository a formal thing and introduced the concept of changesets >> that keeps track of all this stuff at the repository level. So it does >> all this stuff at the file level but you don't have to do that low level >> work. You could think of SCCS as assembly and BitKeeper as more like C, >> it upleveled things to the point that humans can manage the repository >> easily. >> >> Whew. That's a butt load of info. Perhaps better for COFF? Any >> questions? It should be obvious that I *love* SCCS, it's a dramatically >> better file format than a patch based one, you can get *any* version of >> the file in constant time, authorship can be preserved across versions, >> it's pretty brilliant and I consider myself blessed to be posting this >> in response to SCCS's creator. Hats off to Marc. And big boo, hiss, >> to the RCS guy, who got a PhD for RCS (give me a break) and did the >> world a huge disservice by bad mouthing SCCS so he could promote RCS. >> >> --lm >> > > > -- > *My new email address is mrochkind at gmail.com * > -------------- next part -------------- An HTML attachment was scrubbed... URL: From norman at oclsc.org Sat Dec 14 08:33:55 2024 From: norman at oclsc.org (Norman Wilson) Date: Fri, 13 Dec 2024 17:33:55 -0500 (EST) Subject: [TUHS] SCCS roach motel Message-ID: <9838D982B117AE37BCEE88BCF8421F2D.for-standards-violators@oclsc.org> Rob Pike: According to the Unix room fortunes file, the actual quote is SCCS: the source-code motel -- your code checks in but it never checks out. Ken Thompson ==== As a Unix-room-culture aside: I believe this quote was what inspired Andrew Hume to call his backup system the File Motel. Norman Wilson Toronto ON From norman at oclsc.org Sat Dec 14 08:57:55 2024 From: norman at oclsc.org (Norman Wilson) Date: Fri, 13 Dec 2024 17:57:55 -0500 (EST) Subject: [TUHS] SCCS roach motel Message-ID: <242CD757E4871441B72EA52F30CF4531.for-standards-violators@oclsc.org> This is verging on COFF material, and I won't mind if someone moves the discussion thither: Clem Cole: As a satisfied user of SCCS (and later Bitkeeper), it's still my preferred choice. ==== I have to admit SCCS is one of the many pieces of software I tried for a week or two > 40 years ago and abandoned because I couldn't stand it. I don't think I ever tried RCS, because I could see it didn't what I saw as the underlying problems. CVS likewise. Subversion was the earliest version-control system that felt usable to me. What bugged me so much? The earlier systems were focussed entirely (or for CVS, almost entirely) on individual files. There was no way to link changes that affected more than one file: -- SCCS and RCS don't (so far as I can tell) understand any relation between files at all. When I'm working on a real project of any size, there is almost always more than one file, and there are often changes that affect multiple files at once: if an interface changes, the changes to definitions and uses really all should be a single commit, logged in one go and reversible as a single coordinated operation; if I refactor code, moving it between files and perhaps adding files or removing them, likewise. It is possible to check in a bunch of files at once and reuse the log message, but I couldn't find a way to make it a true coordinated single commit; and neither SCCS nor RCS has a way I could find to record, as a structured commit, that files are added or removed from a directory or directory tree. -- CVS can track adds and deletes and can bundle changes into a single commit on the command line, but the changes are still stored individually per file. -- None of the systems even tries to track file renames, something I also do occasionally as programs and especially documentation evolves. It wasn't until I discovered Subversion that I started using version control regularly in my own work. Ironically, a few years after I began using svn for myself, I ended up working in places that had great compost heaps of RCS and CVS. This would have made sense in the 1990s, but it was in the 2010s, and continues in the 2020s. I now use hg for my personal work, and have been attempting to drag my workplace into using git if only so new hires don't have to learn clumsy outdated stuff just to do their jobs. I expect to retire (not for this reason) before I really succeed, though. Norman Wilson Toronto ON From lm at mcvoy.com Sat Dec 14 09:19:47 2024 From: lm at mcvoy.com (Larry McVoy) Date: Fri, 13 Dec 2024 15:19:47 -0800 Subject: [TUHS] SCCS roach motel In-Reply-To: <242CD757E4871441B72EA52F30CF4531.for-standards-violators@oclsc.org> References: <242CD757E4871441B72EA52F30CF4531.for-standards-violators@oclsc.org> Message-ID: <20241213231947.GB11590@mcvoy.com> On Fri, Dec 13, 2024 at 05:57:55PM -0500, Norman Wilson wrote: > This is verging on COFF material, and I won't mind if someone > moves the discussion thither: > > Clem Cole: > > As a satisfied user of SCCS (and later Bitkeeper), it's still my preferred > choice. > > ==== > > I have to admit SCCS is one of the many pieces of software > I tried for a week or two > 40 years ago and abandoned because > I couldn't stand it. I don't think I ever tried RCS, because > I could see it didn't what I saw as the underlying problems. > CVS likewise. Subversion was the earliest version-control > system that felt usable to me. > > What bugged me so much? The earlier systems were focussed > entirely (or for CVS, almost entirely) on individual files. > There was no way to link changes that affected more than one > file: That was the problem that BitKeeper solved. There was an extra step, bk commit, that glued all the files together in an atomic commit. That and each commit was like a CVS tag, you can roll the history back to any commit, no tags are needed. That's because while you think of a revision as 1.5 or whatever, and BitKeeper had that interface, the real name is a a provably unique key made up of user at host|path/to/file.c|time_t|sccs_cksum We called those "keys" and you could use a key any place you could use a revision. I'll spare you how we made them unique, but I can tell you that in two decades of BK use on every continent other than the artic, we've never had a key collision. Does require that you use DNS. BTW, the CVS/SCCS/RCS importers guessed at commit boundaries and made those systems yield commits. We looked at author, check in comments, and time stamps, same author, same comments and within a short window, that's the same commit. --lm From lm at mcvoy.com Sat Dec 14 09:32:03 2024 From: lm at mcvoy.com (Larry McVoy) Date: Fri, 13 Dec 2024 15:32:03 -0800 Subject: [TUHS] [rick@rbsmith.com: Re: SCCS roach motel] Message-ID: <20241213233203.GC11590@mcvoy.com> ----- Forwarded message from Rick Smith ----- Date: Fri, 13 Dec 2024 18:22:29 -0500 From: Rick Smith To: Larry McVoy Subject: Re: [TUHS] SCCS roach motel X-Mailer: Apple Mail (2.3826.300.87.4.3) Hi Larry, I likely can???t post to TUHS without joining, though I can read it fine. Anyway, feel free to edit and post: Marc, I remember in 1992 driving to the UCSD library to get that issue of TSE and make make copy of the article. I still have it and wrote about it on HN [1]. The link I posted at the end there is dead (wayback [2]). It is a post by J??rg Schilling about your post here on TUHS about SCCS. Yes, we could get lost in the weeds about choice of control-A, lack of merge bookkeeping and the corner cases with -i and -x when applying strict order to a partial order when computing the graph-to-set but in the big picture, the SCCS system is a huge contribution. Congratulations on writing one of the most influential papers in TSE's first decade! Certainly is that for me. Aside, Larry wrote: > ... He [Rick] did point out that my weave implementation was the only > one written such that I could have N serial sets in my hand, and do one > pass through the weave and get N different checked out files. I don't > think we ever used that but if we did it would be in smerge.c. The original makepatch() uses it in sccs_getdiffs() which walks the weave calling changestate() once to track weave structure and printstate() twice, generating a diff in one pass. While this is a hats off to Marc, there is also a hats off to Larry for extending the SCCS work with TeamWare and Bitkeeper. I studied SCCS (and other engines). Larry built systems. [1] https://news.ycombinator.com/item?id=26225001 [2] https://web.archive.org/web/20190403213137/http://sccs.sourceforge.net/sccs_invention.html ----- End forwarded message ----- -- --- Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From imp at bsdimp.com Sat Dec 14 09:38:49 2024 From: imp at bsdimp.com (Warner Losh) Date: Fri, 13 Dec 2024 16:38:49 -0700 Subject: [TUHS] SCCS roach motel In-Reply-To: <20241213231947.GB11590@mcvoy.com> References: <242CD757E4871441B72EA52F30CF4531.for-standards-violators@oclsc.org> <20241213231947.GB11590@mcvoy.com> Message-ID: On Fri, Dec 13, 2024, 4:19 PM Larry McVoy wrote: > On Fri, Dec 13, 2024 at 05:57:55PM -0500, Norman Wilson wrote: > > This is verging on COFF material, and I won't mind if someone > > moves the discussion thither: > > > > Clem Cole: > > > > As a satisfied user of SCCS (and later Bitkeeper), it's still my > preferred > > choice. > > > > ==== > > > > I have to admit SCCS is one of the many pieces of software > > I tried for a week or two > 40 years ago and abandoned because > > I couldn't stand it. I don't think I ever tried RCS, because > > I could see it didn't what I saw as the underlying problems. > > CVS likewise. Subversion was the earliest version-control > > system that felt usable to me. > > > > What bugged me so much? The earlier systems were focussed > > entirely (or for CVS, almost entirely) on individual files. > > There was no way to link changes that affected more than one > > file: > > That was the problem that BitKeeper solved. There was an extra step, > bk commit, that glued all the files together in an atomic commit. > That and each commit was like a CVS tag, you can roll the history back > to any commit, no tags are needed. That's because while you think of a > revision as 1.5 or whatever, and BitKeeper had that interface, the real > name is a a provably unique key made up of > > user at host|path/to/file.c|time_t|sccs_cksum > > We called those "keys" and you could use a key any place you could use > a revision. > That's a nice feature... Too bad we don't have it for the historic SCCS trees. Also, the historical SCCS trees lack metatdata about file renames (which were done by moving the ,s files)... Warner I'll spare you how we made them unique, but I can tell you that in two > decades of BK use on every continent other than the artic, we've never > had a key collision. Does require that you use DNS. > > BTW, the CVS/SCCS/RCS importers guessed at commit boundaries and made > those systems yield commits. We looked at author, check in comments, > and time stamps, same author, same comments and within a short window, > that's the same commit. > > --lm > -------------- next part -------------- An HTML attachment was scrubbed... URL: From luther.johnson at makerlisp.com Sat Dec 14 10:37:01 2024 From: luther.johnson at makerlisp.com (Luther Johnson) Date: Fri, 13 Dec 2024 17:37:01 -0700 Subject: [TUHS] SCCS roach motel In-Reply-To: References: Message-ID: <3f27aaff-37e3-598c-62fa-3bf0997bf057@makerlisp.com> When I just read your quote, this joke came to mind: What's the difference between a public library and a software library ? At the public library, everyone wants to check things out, but no-one ever wants to return anything. In a source code library, everyone wants to put something in the library, but programmers rarely want to take anything in it out, and use it. Maybe it doesn't seem as funny as the first time I heard it, but I think it's still true. Maybe a similar thought is behind the 'source motel' quote. On 12/13/2024 09:52 AM, Marc Rochkind wrote: > IEEE Transactions on Software Engineering has asked me to write a > retrospective on the influence of SCCS over the last 50 years, as my > SCCS paper was published in 1975. They consider it one of the most > influential papers from TSE's first decade. > > There's a funny quote from Ken Thompson that circulates from time-to-time: > > "SCCS, the source motel! Programs check in and never check out!" > > But nobody seems to know what it means exactly. As part of my > research, I asked Ken what the quote meant, sunce I wanted to include > it. He explained that it refers to SCCS storing binary data in its > repository file, preventing UNIX text tools from operating on the file. > > Of course, this is only one of SCCS's many weaknesses. If you have > anything funny about any of the others, post it here. I already have > all the boring usual stuff (e.g., long-term locks, file-oriented, no > merging). > > Marc Rochkind > mrochkind.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Sat Dec 14 10:53:05 2024 From: lm at mcvoy.com (Larry McVoy) Date: Fri, 13 Dec 2024 16:53:05 -0800 Subject: [TUHS] SCCS roach motel In-Reply-To: References: <242CD757E4871441B72EA52F30CF4531.for-standards-violators@oclsc.org> <20241213231947.GB11590@mcvoy.com> Message-ID: <20241214005305.GE11590@mcvoy.com> On Fri, Dec 13, 2024 at 04:38:49PM -0700, Warner Losh wrote: > On Fri, Dec 13, 2024, 4:19???PM Larry McVoy wrote: > > > On Fri, Dec 13, 2024 at 05:57:55PM -0500, Norman Wilson wrote: > > > This is verging on COFF material, and I won't mind if someone > > > moves the discussion thither: > > > > > > Clem Cole: > > > > > > As a satisfied user of SCCS (and later Bitkeeper), it's still my > > preferred > > > choice. > > > > > > ==== > > > > > > I have to admit SCCS is one of the many pieces of software > > > I tried for a week or two > 40 years ago and abandoned because > > > I couldn't stand it. I don't think I ever tried RCS, because > > > I could see it didn't what I saw as the underlying problems. > > > CVS likewise. Subversion was the earliest version-control > > > system that felt usable to me. > > > > > > What bugged me so much? The earlier systems were focussed > > > entirely (or for CVS, almost entirely) on individual files. > > > There was no way to link changes that affected more than one > > > file: > > > > That was the problem that BitKeeper solved. There was an extra step, > > bk commit, that glued all the files together in an atomic commit. > > That and each commit was like a CVS tag, you can roll the history back > > to any commit, no tags are needed. That's because while you think of a > > revision as 1.5 or whatever, and BitKeeper had that interface, the real > > name is a a provably unique key made up of > > > > user at host|path/to/file.c|time_t|sccs_cksum > > > > We called those "keys" and you could use a key any place you could use > > a revision. > > > > That's a nice feature... Too bad we don't have it for the historic SCCS > trees. SCCS doesn't version pathnames so you have to fake it. You could look at BitKeeper's import -tSCCS to see what we did, I would think we would have done something sensible. > Also, the historical SCCS trees lack metatdata about file renames (which > were done by moving the ,s files)... See above. BitKeeper versioned not only path names (they are an attribute of each delta) but also file types: symlink, regular, not sure if we ever did hard links. -- --- Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From mrochkind at gmail.com Sat Dec 14 11:11:57 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Fri, 13 Dec 2024 18:11:57 -0700 Subject: [TUHS] SCCS roach motel In-Reply-To: References: Message-ID: On Fri, Dec 13, 2024 at 2:10 PM Dan Cross wrote: > > I had heard a story once, that based on what you wrote today, I now > think is apocryphal and never actually happened. > > The story was that there was a bug in early SCCS where a source file > had to be checked in twice before it could be (successfully) checked > out. The bug was such that if one only checked it in once and tried to > check it out, it would truncate the file. "Programmers using it just > learned to check in twice." > > I never used SCCS extensively, and certainly never observed that > behavior, so chucked it up to the bug having been long fixed. As I > mentioned above, I doubt it was ever there to begin with. A fun story, > though. > > - Dan C. > This is completely ridiculous. No program I wrote ever had one of those bug things. Marc -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Sat Dec 14 11:27:54 2024 From: crossd at gmail.com (Dan Cross) Date: Fri, 13 Dec 2024 20:27:54 -0500 Subject: [TUHS] SCCS roach motel In-Reply-To: References: Message-ID: On Fri, Dec 13, 2024 at 8:12 PM Marc Rochkind wrote: > On Fri, Dec 13, 2024 at 2:10 PM Dan Cross wrote: >> I had heard a story once, that based on what you wrote today, I now >> think is apocryphal and never actually happened. >> >> The story was that there was a bug in early SCCS where a source file >> had to be checked in twice before it could be (successfully) checked >> out. The bug was such that if one only checked it in once and tried to >> check it out, it would truncate the file. "Programmers using it just >> learned to check in twice." >> >> I never used SCCS extensively, and certainly never observed that >> behavior, so chucked it up to the bug having been long fixed. As I >> mentioned above, I doubt it was ever there to begin with. A fun story, >> though. > > This is completely ridiculous. No program I wrote ever had one of those bug things. Oh dear, I do hope no offense was taken. As I said, I firmly believe the story was apocryphal. The identity of the person from whom I heard it is probably known to at least some of the USENIX old school crew; I rather thought at the time they were under the impression that SCCS was the result of a large team, or perhaps some kind of fork. Anyway, I don't mean to belabor the point. Please accept my apologies if the story rubbed you the wrong way. - Dan C. From lm at mcvoy.com Sat Dec 14 11:38:56 2024 From: lm at mcvoy.com (Larry McVoy) Date: Fri, 13 Dec 2024 17:38:56 -0800 Subject: [TUHS] SCCS roach motel In-Reply-To: References: Message-ID: <20241214013856.GI11590@mcvoy.com> On Fri, Dec 13, 2024 at 06:11:57PM -0700, Marc Rochkind wrote: > On Fri, Dec 13, 2024 at 2:10???PM Dan Cross wrote: > > > > > I had heard a story once, that based on what you wrote today, I now > > think is apocryphal and never actually happened. > > > > The story was that there was a bug in early SCCS where a source file > > had to be checked in twice before it could be (successfully) checked > > out. The bug was such that if one only checked it in once and tried to > > check it out, it would truncate the file. "Programmers using it just > > learned to check in twice." > > > > I never used SCCS extensively, and certainly never observed that > > behavior, so chucked it up to the bug having been long fixed. As I > > mentioned above, I doubt it was ever there to begin with. A fun story, > > though. > > > > - Dan C. > > > > This is completely ridiculous. No program I wrote ever had one of those bug > things. Heh. Me too, bug? What's that? BTW, this thread, Marc, has had me wandering into the BitKeeper code over and over, and while I'm done with coding, it was super pleasant to go look at that code. We didn't win but we wrote really nice C. I'm proud of that, it's an old guy thing, it's nice to look back and see we did a good job. From lm at mcvoy.com Sat Dec 14 11:39:43 2024 From: lm at mcvoy.com (Larry McVoy) Date: Fri, 13 Dec 2024 17:39:43 -0800 Subject: [TUHS] SCCS roach motel In-Reply-To: References: Message-ID: <20241214013943.GJ11590@mcvoy.com> On Fri, Dec 13, 2024 at 08:27:54PM -0500, Dan Cross wrote: > On Fri, Dec 13, 2024 at 8:12???PM Marc Rochkind wrote: > > On Fri, Dec 13, 2024 at 2:10???PM Dan Cross wrote: > >> I had heard a story once, that based on what you wrote today, I now > >> think is apocryphal and never actually happened. > >> > >> The story was that there was a bug in early SCCS where a source file > >> had to be checked in twice before it could be (successfully) checked > >> out. The bug was such that if one only checked it in once and tried to > >> check it out, it would truncate the file. "Programmers using it just > >> learned to check in twice." > >> > >> I never used SCCS extensively, and certainly never observed that > >> behavior, so chucked it up to the bug having been long fixed. As I > >> mentioned above, I doubt it was ever there to begin with. A fun story, > >> though. > > > > This is completely ridiculous. No program I wrote ever had one of those bug things. > > Oh dear, I do hope no offense was taken. As I said, I firmly believe > the story was apocryphal. > > The identity of the person from whom I heard it is probably known to > at least some of the USENIX old school crew; I rather thought at the > time they were under the impression that SCCS was the result of a > large team, or perhaps some kind of fork. > > Anyway, I don't mean to belabor the point. Please accept my apologies > if the story rubbed you the wrong way. I'm 90% sure Marc was joking, I got it. From mrochkind at gmail.com Sat Dec 14 16:20:36 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Fri, 13 Dec 2024 23:20:36 -0700 Subject: [TUHS] SCCS roach motel In-Reply-To: References: Message-ID: That was a joke, Dan. I've been programming for 58 years, and nobody has put in more bugs than me, I'm sure. Marc On Fri, Dec 13, 2024 at 6:28 PM Dan Cross wrote: > On Fri, Dec 13, 2024 at 8:12 PM Marc Rochkind wrote: > > On Fri, Dec 13, 2024 at 2:10 PM Dan Cross wrote: > >> I had heard a story once, that based on what you wrote today, I now > >> think is apocryphal and never actually happened. > >> > >> The story was that there was a bug in early SCCS where a source file > >> had to be checked in twice before it could be (successfully) checked > >> out. The bug was such that if one only checked it in once and tried to > >> check it out, it would truncate the file. "Programmers using it just > >> learned to check in twice." > >> > >> I never used SCCS extensively, and certainly never observed that > >> behavior, so chucked it up to the bug having been long fixed. As I > >> mentioned above, I doubt it was ever there to begin with. A fun story, > >> though. > > > > This is completely ridiculous. No program I wrote ever had one of those > bug things. > > Oh dear, I do hope no offense was taken. As I said, I firmly believe > the story was apocryphal. > > The identity of the person from whom I heard it is probably known to > at least some of the USENIX old school crew; I rather thought at the > time they were under the impression that SCCS was the result of a > large team, or perhaps some kind of fork. > > Anyway, I don't mean to belabor the point. Please accept my apologies > if the story rubbed you the wrong way. > > - Dan C. > -- *My new email address is mrochkind at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From lars at nocrew.org Sat Dec 14 17:20:58 2024 From: lars at nocrew.org (Lars Brinkhoff) Date: Sat, 14 Dec 2024 07:20:58 +0000 Subject: [TUHS] BSD talk program? In-Reply-To: (Dan Cross's message of "Fri, 13 Dec 2024 09:28:38 -0500") References: Message-ID: <7wplluloh1.fsf@junk.nocrew.org> Dan Cross wrote: > I'm curious if anyone has any history they can share about the BSD > "talk" program. Allow me to interject for a moment. Apparently there's a ttylinkd program that "is a simple daemon that allows incoming ttylink calls to be routed through to Linux's normal talkd(8) system". It's TCP port 87 in RFC 1060 "Assigned Numbers", but seems to have been dropped since. Maybe someone can expand on this. Apparently this protocol was known at MIT, because there's a Chaosnet servic called TTYLINK. The client side is implemented by the MINITS Chaosnet router/terminal concentrator/etc. From clemc at ccc.com Sun Dec 15 01:40:38 2024 From: clemc at ccc.com (Clem Cole) Date: Sat, 14 Dec 2024 10:40:38 -0500 Subject: [TUHS] BSD Talk History -- Credit Where Credit is Due Message-ID: I was thinking about this some more. IIRC: Peter and I sketched out the protocol for the sockets version on a whiteboard in our office one night after a beer and pizza run. Rick Spicklemeir, Tom Quarles, and Jim Kleckner also participated in those bull sessions. I started writing the program soon after that and had it working to a point in a couple of hours. I don't remember the issues, but a couple of them were when I left for the USENIX conference later that week. When I got back Peter had finished it and put it into RCS. The key is that the coding was primarily Peter and myself, but Rick, TQ, and Jim all had contributed in some manner, too, Although the famous bug of using a vax integer, you can squarely blame me — and as I said, having worked on networking for several years before my time at UCB, I should have known better. But did not even think about it. I failed Henry's ten programming commandments and concluded that the world was a Vax. Mei culpa. ᐧ -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Sun Dec 15 02:00:56 2024 From: crossd at gmail.com (Dan Cross) Date: Sat, 14 Dec 2024 11:00:56 -0500 Subject: [TUHS] BSD Talk History -- Credit Where Credit is Due In-Reply-To: References: Message-ID: On Sat, Dec 14, 2024 at 10:41 AM Clem Cole wrote: > I was thinking about this some more. > > IIRC: Peter and I sketched out the protocol for the sockets version on a whiteboard in our office one night after a beer and pizza run. Rick Spicklemeir, Tom Quarles, and Jim Kleckner also participated in those bull sessions. I started writing the program soon after that and had it working to a point in a couple of hours. I don't remember the issues, but a couple of them were when I left for the USENIX conference later that week. When I got back Peter had finished it and put it into RCS. The key is that the coding was primarily Peter and myself, but Rick, TQ, and Jim all had contributed in some manner, too, > > Although the famous bug of using a vax integer, you can squarely blame me — and as I said, having worked on networking for several years before my time at UCB, I should have known better. But did not even think about it. I failed Henry's ten programming commandments and concluded that the world was a Vax. Mei culpa. > ᐧ Thanks, Clem, these are very interesting notes. The protocol had some interesting aspects to it; since the ctl address was embedded in the message sent to the distant end, a trick of some locals when I was younger was to put fake data in the request. The effect would be that one would get a request to talk from user1 at host1, but actually be talking to user2 at host2. This could either be very funny or very uncool. I'm curious how the original worked, which I sort of gather was before sockets? How did the two users rendezvous? - Dan C. From clemc at ccc.com Sun Dec 15 02:35:39 2024 From: clemc at ccc.com (Clem Cole) Date: Sat, 14 Dec 2024 11:35:39 -0500 Subject: [TUHS] BSD Talk History -- Credit Where Credit is Due In-Reply-To: References: Message-ID: I don't remember how Kipp did it. It would have been very V7 oriented and using the FS and signals I suspect. I remember is was not very sexy and one of things sockets allowed was completely rethink. That was the bull session I was referring too. As for the security issue, sure - we would not have considered that. Again we were grad students and basically friendly so actions like that would have been considered uncool. As I have pointed out elsewhere when a connection to the ARPAnet cost 250K/yr per host and a host cost $1-4M, the concepts of security and us all being in it together—a friendly community, as it were—had different behaviors when a host is is $10-30 micro and wireless Interconnect can slimed at Starbucks for free. ᐧ On Sat, Dec 14, 2024 at 11:01 AM Dan Cross wrote: > On Sat, Dec 14, 2024 at 10:41 AM Clem Cole wrote: > > I was thinking about this some more. > > > > IIRC: Peter and I sketched out the protocol for the sockets version on a > whiteboard in our office one night after a beer and pizza run. Rick > Spicklemeir, Tom Quarles, and Jim Kleckner also participated in those bull > sessions. I started writing the program soon after that and had it working > to a point in a couple of hours. I don't remember the issues, but a couple > of them were when I left for the USENIX conference later that week. When I > got back Peter had finished it and put it into RCS. The key is that the > coding was primarily Peter and myself, but Rick, TQ, and Jim all had > contributed in some manner, too, > > > > Although the famous bug of using a vax integer, you can squarely blame > me — and as I said, having worked on networking for several years before my > time at UCB, I should have known better. But did not even think about it. > I failed Henry's ten programming commandments and concluded that the world > was a Vax. Mei culpa. > > ᐧ > > Thanks, Clem, these are very interesting notes. > > The protocol had some interesting aspects to it; since the ctl address > was embedded in the message sent to the distant end, a trick of some > locals when I was younger was to put fake data in the request. The > effect would be that one would get a request to talk from user1 at host1, > but actually be talking to user2 at host2. This could either be very > funny or very uncool. > > I'm curious how the original worked, which I sort of gather was before > sockets? How did the two users rendezvous? > > - Dan C. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Sun Dec 15 02:46:31 2024 From: crossd at gmail.com (Dan Cross) Date: Sat, 14 Dec 2024 11:46:31 -0500 Subject: [TUHS] BSD talk program? In-Reply-To: <7wplluloh1.fsf@junk.nocrew.org> References: <7wplluloh1.fsf@junk.nocrew.org> Message-ID: On Sat, Dec 14, 2024 at 2:21 AM Lars Brinkhoff wrote: > Dan Cross wrote: > > I'm curious if anyone has any history they can share about the BSD > > "talk" program. > > Allow me to interject for a moment. Apparently there's a ttylinkd > program that "is a simple daemon that allows incoming ttylink calls to > be routed through to Linux's normal talkd(8) system". The only `ttylinkd` that I'm aware of is the one that's part of the Linux AX.25/NETROM/Rose suite. This lets a ham set up a service for one of those protocols; I set this up for connected-mode AX.25 at my QTH (`KZ2X-2` on 144.09 MHz in the Boston area). In a nutshell, if someone uses AX.25 and connects to that SSID, the system invokes the ttylinkd daemon, which knows enough about the `talk` protocol to connect to a talk daemon and send an invitation to a (fixed) user. That user can use any normal `talk` client to talk to the user on the radio end, allowing keyboard-to-keyboard communication. It's not character-by-character the way that normal "talk" is because AX.25 is in the way and generally wants to batch up keystrokes into lines/packets (it's slow), but it's kinda cool. Of course, no one around here uses it. :-( > It's TCP port 87 > in RFC 1060 "Assigned Numbers", but seems to have been dropped since. > Maybe someone can expand on this. > > Apparently this protocol was known at MIT, because there's a Chaosnet > servic called TTYLINK. The client side is implemented by the MINITS > Chaosnet router/terminal concentrator/etc. I suspect this is something different, and the name collision is just an unfortunate coincidence. - Dan C. From mrochkind at gmail.com Sun Dec 15 03:43:47 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Sat, 14 Dec 2024 10:43:47 -0700 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git Message-ID: As I mentioned in another post, I'm writing an invited paper for an upcoming issue of IEEE Transactions on Software Engineering that will be a 50-year retrospective of my original 1975 SCCS paper ( mrochkind.com/aup/talks/SCCS-Slideshow.pdf). Can some people here review a couple of paragraphs for accuracy? *Decentralized Version Control (DVCS)* *While VCSs like CVS and Subversion were centralized and had pre-commit merging, a further advance was towards decentralization, with post-commit merging. Probably the first DVCS was Sun WorkShop TeamWare, created by Larry McVoy and announced in 1992 [sun]. It was implemented as a layer on top of SCCS. McVoy later commercialized a successor system called BitKeeper [Bitkeeper], which was layered on a re-implementation of SCCS, which he called BitSCCS. TeamWare and BitKeeper took advantage of the interleaved delta algorithm, also known as a weave, to implement an efficient way to represent merged deltas by reference, instead of reproducing code inside the repository. This is a lot more complicated to do with reverse deltas, introduced by RCS.* *In 2005 Linus Torvalds, creator of Linux [linux], invented the DVCS Git [git] for Linux development, and since then Git has become widely used and has supplanted BitKeeper.* [more about DVCS follows] I don't want to add more detail that would make these paragraphs any longer, but I do want them to be accurate. Thanks! Marc Rochkind -- *My new email address is mrochkind at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Sun Dec 15 04:04:20 2024 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sat, 14 Dec 2024 11:04:20 -0700 Subject: [TUHS] BSD talk program? In-Reply-To: <20241213155336.GV11590@mcvoy.com> References: <20241213155336.GV11590@mcvoy.com> Message-ID: <202412141804.4BEI4K5c1726899@freefriends.org> When my wife and I were dating, she was at U Penn in Philadelphia and I was at Emory in Atlanta. We occasionally used talk to chat in real time. :-) This was in 1989. Larry McVoy wrote: > I loved talk when CS was running BSD on a VAX. You could see who was on > and talk them. Very handy and it was sort of social. > > It's crazy how things were back then, open ports listening for all sorts > of things. I think we were pretty unaware of how nasty the internet would > get. > > On Fri, Dec 13, 2024 at 10:22:22AM -0500, Clem Cole wrote: > > As for the motivation -- it was simple. UCB is on a hill. I lived at the > > base of hill and I only wanted to walk up it once a day. Our office was a > > big pool of about 20 of us next to the CAD machine room on the second floor > > of Cory Hall. Somebody was usually in the office most nights, but not > > everyone. We all had modems and terminals at home, but only one phone > > line. We had 3 Vaxes in the CAD group, plus my Array Processor. So I > > wanted to be able to ask someone like Peter or TQ to reset the AP for me if > > I hosed it when I was working from home when I was debugging it. Plus > > the obvious social aspects -- "hey you want go get a Pizza/Beer etc..." > > But since we might be working on a different system, Kipps' hack was > > useless. > > ??? > > ??? > > ??? > > ??? > > > > On Fri, Dec 13, 2024 at 10:14???AM Clem Cole wrote: > > > > > Yes -- I can give this history. > > > Kipp wrote an early version for 4.1BSD - but it is not the version in the > > > releases. It ran on Ernie and did not do as much. > > > I had used a different program on the PDP-10's and the ARPANET and I > > > started over when Joy added sockets for 4.1A. I also made the infamous use > > > of vax integers instead of network integers (and I knew better - but really > > > did not think about until a few years later when I was at Masscomp and > > > compiled it for the 68000 -- ugh). That version still had a couple of bugs > > > in it (i.e. hung in the 4.1A networking code occasionally), but worked well > > > enough on the CAD systems. I went away to a USENIX conference and while I > > > was gone, my officemate Peter (Moore) took my code and fixed the problem, > > > plus he put it into RCS. I gave that to Sam and that's the version that > > > went out in 4.1C and beyond. > > > > > > Clem > > > > > > > > > ??? > > > ??? > > > > > > On Fri, Dec 13, 2024 at 9:29???AM Dan Cross wrote: > > > > > >> I'm curious if anyone has any history they can share about the BSD > > >> "talk" program. > > >> > > >> I was fond of this back when it was still (relatively) common, but > > >> given the way it's architected I definitely see why it fell out of use > > >> as the Internet grew. Still, does anybody know what the history behind > > >> it is? Initially, I thought it was written by Mike Karels, but that > > >> was just my speculation from SCCS spelunking, and looking at the > > >> sources from 4.2, I see RCS header strings that indicate it was > > >> written by "moore" (Peter Moore?). talk.c says, "Written by Kipp > > >> Hickman". > > >> > > >> It seems to have arrived pretty early on with respect to the > > >> introduction of TCP/IP in BSD: the README alludes to some things > > >> coming up in 4.1c. Clem, you seem to have had a hand in it, and are > > >> credited (along with Peter Moore) for making it work on 4.1a. > > >> > > >> So I guess the question is, what was the motivation? Was it just to > > >> have a more pleasing user-to-user communications experience, or was > > >> discussion across the network an explicit goal? There's a note in > > >> talk.c ("Modified to run between hosts by Peter Moore, 8/19/82") that > > >> suggests this wasn't the original intent. Who thought up the > > >> character-at-a-time display mode? > > >> > > >> Thanks for any insights. > > >> > > >> - Dan C. > > >> > > > > > -- > --- > Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From g.branden.robinson at gmail.com Sun Dec 15 04:25:56 2024 From: g.branden.robinson at gmail.com (G. Branden Robinson) Date: Sat, 14 Dec 2024 12:25:56 -0600 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: <20241214182556.xdgvuc2zw4bkj4v7@illithid> At 2024-12-14T10:43:47-0700, Marc Rochkind wrote: > As I mentioned in another post, I'm writing an invited paper for an > upcoming issue of IEEE Transactions on Software Engineering that will > be a 50-year retrospective of my original 1975 SCCS paper ( > mrochkind.com/aup/talks/SCCS-Slideshow.pdf). Can some people here > review a couple of paragraphs for accuracy? I apologize for this not being quite responsive to your query, then. 1. Tom Lord's Arch (ca. 2001) was at one timely recognized for popularizing, or perhaps _introducing_, the notion of decentralized version control to the dot-com era of hard-scrabble FLOSS developers who were building "Web 2.0" and whose startup employers, be they flush with cash or not, generally would not spring for a commercial VCS (which might not have been decentralized anyway). The most salient point here is that Linus Torvalds was not prescient or uniquely perceptive in recognizing the value of a DVCS. Many people in the first half of the decade of the 2000s were chafing at the limitations of CVS in an increasingly networked environment, and Subversion was recognized as a conservative alternative even at the time. This was conventional wisdom among engineers at my workplace. I urge us not to contribute to Torvalds's cult of celebrity as we did (and do) that of Bill Joy or Steve Jobs, suggesting that they had unique insights when they plainly didn't. Sometimes a big name or reputation can lend some activation energy to an existing good idea (or persuade someone with a fat wallet to open it for financing of development), but it's important not to mistake what such celebrities should be credited with. An example is how people mis-credit Bill Joy, solely, with developing curses.[1] 2. "TeamWare and BitKeeper took advantage of the interleaved delta algorithm, also known as a weave, to implement an efficient way to represent merged deltas by reference, instead of reproducing code inside the repository. This is a lot more complicated to do with reverse deltas, introduced by RCS.*" I'd a like a second footnote directing me to where I can understand the mathematics supporting this claim. Just out of nerd interest. Also, if one requires a shovel to bury Walter Tichy with, such a presentation would serve well... Regards, Branden [1] See, e.g., the section "Relationship with vi": https://invisible-island.net/ncurses/ncurses.faq.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From arnold at skeeve.com Sun Dec 15 04:29:45 2024 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sat, 14 Dec 2024 11:29:45 -0700 Subject: [TUHS] SCCS roach motel In-Reply-To: References: <20241213180649.GW11590@mcvoy.com> <20241213185534.GY11590@mcvoy.com> Message-ID: <202412141829.4BEITjPU1728398@freefriends.org> Henry Bent wrote: > On Fri, 13 Dec 2024 at 14:03, Larry McVoy wrote: > > > On Fri, Dec 13, 2024 at 11:32:57AM -0700, Marc Rochkind wrote: > > > > > Was your implementation of SCCS ever released by itself? > > > > See if the wayback machine has BitSCCS somewhere. It was pretty early, > > before Rick showed up to fix my screwups. He did point out that my > > weave implementation was the only one written such that I could have > > N serial sets in my hand, and do one pass through the weave and get > > N different checked out files. I don't think we ever used that but > > if we did it would be in smerge.c. > > > > There are many preservations of http://www.bitmover.com/bitsccs/, but since > the BitSCCS sources were distributed via FTP the wayback machine doesn't > have the actual sources. > > I did a bit of searching and found > ftp://linux.mathematik.tu-darmstadt.de/pub/linux/distributions/historic/jurix/source/compile/BitSCCS-0.5.2.tar.gz > ; I don't know where that falls in the product's lifetime. > > -Henry Larry, What about GNU CSSC? (https://ftp.gnu.org/gnu/cssc/)? Isn't that a reimplementation of SCCS? Hmm, the README and THANKS file in it provide some info. It's not a full clone, there are some missing features. The last release is from 2019. Arnold From lm at mcvoy.com Sun Dec 15 04:53:19 2024 From: lm at mcvoy.com (Larry McVoy) Date: Sat, 14 Dec 2024 10:53:19 -0800 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241214182556.xdgvuc2zw4bkj4v7@illithid> References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> Message-ID: <20241214185319.GQ11590@mcvoy.com> On Sat, Dec 14, 2024 at 12:25:56PM -0600, G. Branden Robinson wrote: > At 2024-12-14T10:43:47-0700, Marc Rochkind wrote: > > As I mentioned in another post, I'm writing an invited paper for an > > upcoming issue of IEEE Transactions on Software Engineering that will > > be a 50-year retrospective of my original 1975 SCCS paper ( > > mrochkind.com/aup/talks/SCCS-Slideshow.pdf). Can some people here > > review a couple of paragraphs for accuracy? > > I apologize for this not being quite responsive to your query, then. > > 1. Tom Lord's Arch (ca. 2001) was at one timely recognized for > popularizing, or perhaps _introducing_, the notion of decentralized > version control to the dot-com era of hard-scrabble FLOSS developers > who were building "Web 2.0" and whose startup employers, be they > flush with cash or not, generally would not spring for a commercial > VCS (which might not have been decentralized anyway). BitKeeper was 3 years into development by then. Here is the first BK changeset: ChangeSet at 1.1, 1999-03-19 16:11:26-08:00, lm at lm.bitmover.com BitKeeper gets stored in BitKeeper. > The most salient point here is that Linus Torvalds was not prescient > or uniquely perceptive in recognizing the value of a DVCS. Couldn't agree more. He, Dave Miller, and Richard Henderson came to my house in SF because Dave was threatening to fork Linux because Linus used no SCM at all. That pushed all the merges to the next level, to people like Dave. It took me 4-5 hours of explaining how distributed SCM would work for Linus to get it. My wife was unimpressed with him. Whatever, at the end of that day, he said "build that and I'll use it". And we did, he did, and rate of development doubled (according to the people who hated me) and was actually about 10x faster (according to the people who could do math). > 2. "TeamWare and BitKeeper took advantage of the interleaved delta > algorithm, also known as a weave, to implement an efficient way to > represent merged deltas by reference, instead of reproducing code > inside the repository. This is a lot more complicated to do with > reverse deltas, introduced by RCS.*" > > I'd a like a second footnote directing me to where I can understand > the mathematics supporting this claim. Just out of nerd interest. See if this helps: https://www.tuhs.org/pipermail/tuhs/2024-December/031188.html You can work out that SCCS/BK are doing what they claim by running bk annotate on a file that was authored by X but automerged by Y. The authorship stays the same across the merge, that's not true in most SCM systems that copy the data from the branch to the trunk. Happy to answer any questions. And, yes, git is a ginormous step backwards. Only one graph, BK had a graph per file so the GCA is always the correct one, none of this try different ones until you get one that automerges; since there is only one graph, all you get are commit comments, you lose the oh-so-valuable-file-comments that you need when the crap hits the fan; you lose a boat load of performance, especially in areas like annotate/blame when the repo gets big, etc. Git sucks, it really does. Linus claims he wrote it in a week and it shows. -- --- Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From lm at mcvoy.com Sun Dec 15 04:59:51 2024 From: lm at mcvoy.com (Larry McVoy) Date: Sat, 14 Dec 2024 10:59:51 -0800 Subject: [TUHS] SCCS roach motel In-Reply-To: <202412141829.4BEITjPU1728398@freefriends.org> References: <20241213180649.GW11590@mcvoy.com> <20241213185534.GY11590@mcvoy.com> <202412141829.4BEITjPU1728398@freefriends.org> Message-ID: <20241214185951.GR11590@mcvoy.com> On Sat, Dec 14, 2024 at 11:29:45AM -0700, arnold at skeeve.com wrote: > Henry Bent wrote: > > > On Fri, 13 Dec 2024 at 14:03, Larry McVoy wrote: > > > > > On Fri, Dec 13, 2024 at 11:32:57AM -0700, Marc Rochkind wrote: > > > > > > > Was your implementation of SCCS ever released by itself? > > > > > > See if the wayback machine has BitSCCS somewhere. It was pretty early, > > > before Rick showed up to fix my screwups. He did point out that my > > > weave implementation was the only one written such that I could have > > > N serial sets in my hand, and do one pass through the weave and get > > > N different checked out files. I don't think we ever used that but > > > if we did it would be in smerge.c. > > > > > > > There are many preservations of http://www.bitmover.com/bitsccs/, but since > > the BitSCCS sources were distributed via FTP the wayback machine doesn't > > have the actual sources. > > > > I did a bit of searching and found > > ftp://linux.mathematik.tu-darmstadt.de/pub/linux/distributions/historic/jurix/source/compile/BitSCCS-0.5.2.tar.gz > > ; I don't know where that falls in the product's lifetime. > > > > -Henry > > Larry, > > What about GNU CSSC? (https://ftp.gnu.org/gnu/cssc/)? Isn't that a > reimplementation of SCCS? It is, it's C++ and really slow. I looked at it to see if I could skip reimplementing SCCS and decided I couldn't. The problem with SCM in general, is it isn't sexy so it doesn't attract the best people. I was a kernel engineer who got distracted by SCM. I worked on performance, the Sun SCM group had tried to clone clearclase, it was so slow that my friends were leaving Sun. I looked at what it was doing and decided it was unfixable. I realized it was using SCCS under the covers and I wrote NSElite (the slow thing was "NSE") and did the magic such that I could clone/pull/push from my stuff to NSE. The kernel group promptly dumped NSE and moved to NSElite. Sun wanted me to go to the SCM group so that everything was politically correct, I looked at the caliber of people there and declined, it was a definite step down from working on the kernel. BK is what you get when you let a kernel engineer do SCM. From mrochkind at gmail.com Sun Dec 15 05:13:40 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Sat, 14 Dec 2024 12:13:40 -0700 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241214185319.GQ11590@mcvoy.com> References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> Message-ID: Thanks, Larry and Branden. I'll try to work what you're saying into my paper. Larry, I found your post of a few days ago ( https://www.tuhs.org/pipermail/tuhs/2024-December/031188.html) very illuminating, and I'd like to reference it, which I could do with that post. But, is there some other, more referenceable article saying the same thing that I can reference instead? Regarding BK vs. Git. Even if BK is superior, can today's developers use BK? My understanding is that it is no longer supported. Is there a DVCS out there that's an alternative to Git? The reason why this part of my paper has to be short is that I'm discussing the influence of SCCS on software engineering, and its influence on DVCSs is very indirect. My paper isn't a history of VCSs. Marc On Sat, Dec 14, 2024 at 11:53 AM Larry McVoy wrote: > On Sat, Dec 14, 2024 at 12:25:56PM -0600, G. Branden Robinson wrote: > > At 2024-12-14T10:43:47-0700, Marc Rochkind wrote: > > > As I mentioned in another post, I'm writing an invited paper for an > > > upcoming issue of IEEE Transactions on Software Engineering that will > > > be a 50-year retrospective of my original 1975 SCCS paper ( > > > mrochkind.com/aup/talks/SCCS-Slideshow.pdf). Can some people here > > > review a couple of paragraphs for accuracy? > > > > I apologize for this not being quite responsive to your query, then. > > > > 1. Tom Lord's Arch (ca. 2001) was at one timely recognized for > > popularizing, or perhaps _introducing_, the notion of decentralized > > version control to the dot-com era of hard-scrabble FLOSS developers > > who were building "Web 2.0" and whose startup employers, be they > > flush with cash or not, generally would not spring for a commercial > > VCS (which might not have been decentralized anyway). > > BitKeeper was 3 years into development by then. Here is the first BK > changeset: > > ChangeSet at 1.1, 1999-03-19 16:11:26-08:00, lm at lm.bitmover.com > BitKeeper gets stored in BitKeeper. > > > The most salient point here is that Linus Torvalds was not prescient > > or uniquely perceptive in recognizing the value of a DVCS. > > Couldn't agree more. He, Dave Miller, and Richard Henderson came to my > house in SF because Dave was threatening to fork Linux because Linus > used no SCM at all. That pushed all the merges to the next level, to > people like Dave. It took me 4-5 hours of explaining how distributed > SCM would work for Linus to get it. My wife was unimpressed with him. > Whatever, at the end of that day, he said "build that and I'll use it". > And we did, he did, and rate of development doubled (according to the > people who hated me) and was actually about 10x faster (according to the > people who could do math). > > > 2. "TeamWare and BitKeeper took advantage of the interleaved delta > > algorithm, also known as a weave, to implement an efficient way to > > represent merged deltas by reference, instead of reproducing code > > inside the repository. This is a lot more complicated to do with > > reverse deltas, introduced by RCS.*" > > > > I'd a like a second footnote directing me to where I can understand > > the mathematics supporting this claim. Just out of nerd interest. > > See if this helps: > > https://www.tuhs.org/pipermail/tuhs/2024-December/031188.html > > You can work out that SCCS/BK are doing what they claim by running > bk annotate on a file that was authored by X but automerged by Y. > The authorship stays the same across the merge, that's not true in > most SCM systems that copy the data from the branch to the trunk. > > Happy to answer any questions. And, yes, git is a ginormous step > backwards. Only one graph, BK had a graph per file so the GCA is > always the correct one, none of this try different ones until you > get one that automerges; since there is only one graph, all you > get are commit comments, you lose the oh-so-valuable-file-comments > that you need when the crap hits the fan; you lose a boat load of > performance, especially in areas like annotate/blame when the repo > gets big, etc. Git sucks, it really does. Linus claims he wrote > it in a week and it shows. > -- > --- > Larry McVoy Retired to fishing > http://www.mcvoy.com/lm/boat > -- *My new email address is mrochkind at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc.donner at gmail.com Sun Dec 15 05:16:13 2024 From: marc.donner at gmail.com (Marc Donner) Date: Sat, 14 Dec 2024 14:16:13 -0500 Subject: [TUHS] BSD talk program? In-Reply-To: <20241213155336.GV11590@mcvoy.com> References: <20241213155336.GV11590@mcvoy.com> Message-ID: Egad! Back in the day all of the /dev/tty entries were world writable. I used to troll CMU CS Rogue players (when I wasn't playing) from time to time by sending a string to their tty that grabbed the cursor, moved it to where the messages appeared (upper left corner?) and typing "The dangling modifier struck!" That's all it did. I got a lot of pleasure when I overheard two faculty members comparing notes on their experiences with the dangling modifier. Best, Marc ===== nygeek.net mindthegapdialogs.com/home On Fri, Dec 13, 2024 at 10:53 AM Larry McVoy wrote: > I loved talk when CS was running BSD on a VAX. You could see who was on > and talk them. Very handy and it was sort of social. > > It's crazy how things were back then, open ports listening for all sorts > of things. I think we were pretty unaware of how nasty the internet would > get. > > On Fri, Dec 13, 2024 at 10:22:22AM -0500, Clem Cole wrote: > > As for the motivation -- it was simple. UCB is on a hill. I lived at > the > > base of hill and I only wanted to walk up it once a day. Our office > was a > > big pool of about 20 of us next to the CAD machine room on the second > floor > > of Cory Hall. Somebody was usually in the office most nights, but not > > everyone. We all had modems and terminals at home, but only one phone > > line. We had 3 Vaxes in the CAD group, plus my Array Processor. So I > > wanted to be able to ask someone like Peter or TQ to reset the AP for me > if > > I hosed it when I was working from home when I was debugging it. Plus > > the obvious social aspects -- "hey you want go get a Pizza/Beer etc..." > > But since we might be working on a different system, Kipps' hack was > > useless. > > ??? > > ??? > > ??? > > ??? > > > > On Fri, Dec 13, 2024 at 10:14???AM Clem Cole wrote: > > > > > Yes -- I can give this history. > > > Kipp wrote an early version for 4.1BSD - but it is not the version in > the > > > releases. It ran on Ernie and did not do as much. > > > I had used a different program on the PDP-10's and the ARPANET and I > > > started over when Joy added sockets for 4.1A. I also made the infamous > use > > > of vax integers instead of network integers (and I knew better - but > really > > > did not think about until a few years later when I was at Masscomp and > > > compiled it for the 68000 -- ugh). That version still had a couple of > bugs > > > in it (i.e. hung in the 4.1A networking code occasionally), but worked > well > > > enough on the CAD systems. I went away to a USENIX conference and > while I > > > was gone, my officemate Peter (Moore) took my code and fixed the > problem, > > > plus he put it into RCS. I gave that to Sam and that's the version > that > > > went out in 4.1C and beyond. > > > > > > Clem > > > > > > > > > ??? > > > ??? > > > > > > On Fri, Dec 13, 2024 at 9:29???AM Dan Cross wrote: > > > > > >> I'm curious if anyone has any history they can share about the BSD > > >> "talk" program. > > >> > > >> I was fond of this back when it was still (relatively) common, but > > >> given the way it's architected I definitely see why it fell out of use > > >> as the Internet grew. Still, does anybody know what the history behind > > >> it is? Initially, I thought it was written by Mike Karels, but that > > >> was just my speculation from SCCS spelunking, and looking at the > > >> sources from 4.2, I see RCS header strings that indicate it was > > >> written by "moore" (Peter Moore?). talk.c says, "Written by Kipp > > >> Hickman". > > >> > > >> It seems to have arrived pretty early on with respect to the > > >> introduction of TCP/IP in BSD: the README alludes to some things > > >> coming up in 4.1c. Clem, you seem to have had a hand in it, and are > > >> credited (along with Peter Moore) for making it work on 4.1a. > > >> > > >> So I guess the question is, what was the motivation? Was it just to > > >> have a more pleasing user-to-user communications experience, or was > > >> discussion across the network an explicit goal? There's a note in > > >> talk.c ("Modified to run between hosts by Peter Moore, 8/19/82") that > > >> suggests this wasn't the original intent. Who thought up the > > >> character-at-a-time display mode? > > >> > > >> Thanks for any insights. > > >> > > >> - Dan C. > > >> > > > > > -- > --- > Larry McVoy Retired to fishing > http://www.mcvoy.com/lm/boat > -------------- next part -------------- An HTML attachment was scrubbed... URL: From imp at bsdimp.com Sun Dec 15 05:26:43 2024 From: imp at bsdimp.com (Warner Losh) Date: Sat, 14 Dec 2024 12:26:43 -0700 Subject: [TUHS] BSD talk program? In-Reply-To: <202412141804.4BEI4K5c1726899@freefriends.org> References: <20241213155336.GV11590@mcvoy.com> <202412141804.4BEI4K5c1726899@freefriends.org> Message-ID: I tried to talk to my friends at New Mexico Tech when I was working at the Wollongong Group in Palo Alto in 89. It sucked too bad to do since TWG had a badle congested 9600 baud line... Warner On Sat, Dec 14, 2024, 11:04 AM wrote: > When my wife and I were dating, she was at U Penn in Philadelphia > and I was at Emory in Atlanta. We occasionally used talk to > chat in real time. :-) This was in 1989. > > Larry McVoy wrote: > > > I loved talk when CS was running BSD on a VAX. You could see who was on > > and talk them. Very handy and it was sort of social. > > > > It's crazy how things were back then, open ports listening for all sorts > > of things. I think we were pretty unaware of how nasty the internet > would > > get. > > > > On Fri, Dec 13, 2024 at 10:22:22AM -0500, Clem Cole wrote: > > > As for the motivation -- it was simple. UCB is on a hill. I lived at > the > > > base of hill and I only wanted to walk up it once a day. Our office > was a > > > big pool of about 20 of us next to the CAD machine room on the second > floor > > > of Cory Hall. Somebody was usually in the office most nights, but not > > > everyone. We all had modems and terminals at home, but only one phone > > > line. We had 3 Vaxes in the CAD group, plus my Array Processor. So I > > > wanted to be able to ask someone like Peter or TQ to reset the AP for > me if > > > I hosed it when I was working from home when I was debugging it. Plus > > > the obvious social aspects -- "hey you want go get a Pizza/Beer etc..." > > > But since we might be working on a different system, Kipps' hack was > > > useless. > > > ??? > > > ??? > > > ??? > > > ??? > > > > > > On Fri, Dec 13, 2024 at 10:14???AM Clem Cole wrote: > > > > > > > Yes -- I can give this history. > > > > Kipp wrote an early version for 4.1BSD - but it is not the version > in the > > > > releases. It ran on Ernie and did not do as much. > > > > I had used a different program on the PDP-10's and the ARPANET and I > > > > started over when Joy added sockets for 4.1A. I also made the > infamous use > > > > of vax integers instead of network integers (and I knew better - but > really > > > > did not think about until a few years later when I was at Masscomp > and > > > > compiled it for the 68000 -- ugh). That version still had a couple > of bugs > > > > in it (i.e. hung in the 4.1A networking code occasionally), but > worked well > > > > enough on the CAD systems. I went away to a USENIX conference and > while I > > > > was gone, my officemate Peter (Moore) took my code and fixed the > problem, > > > > plus he put it into RCS. I gave that to Sam and that's the version > that > > > > went out in 4.1C and beyond. > > > > > > > > Clem > > > > > > > > > > > > ??? > > > > ??? > > > > > > > > On Fri, Dec 13, 2024 at 9:29???AM Dan Cross > wrote: > > > > > > > >> I'm curious if anyone has any history they can share about the BSD > > > >> "talk" program. > > > >> > > > >> I was fond of this back when it was still (relatively) common, but > > > >> given the way it's architected I definitely see why it fell out of > use > > > >> as the Internet grew. Still, does anybody know what the history > behind > > > >> it is? Initially, I thought it was written by Mike Karels, but that > > > >> was just my speculation from SCCS spelunking, and looking at the > > > >> sources from 4.2, I see RCS header strings that indicate it was > > > >> written by "moore" (Peter Moore?). talk.c says, "Written by Kipp > > > >> Hickman". > > > >> > > > >> It seems to have arrived pretty early on with respect to the > > > >> introduction of TCP/IP in BSD: the README alludes to some things > > > >> coming up in 4.1c. Clem, you seem to have had a hand in it, and are > > > >> credited (along with Peter Moore) for making it work on 4.1a. > > > >> > > > >> So I guess the question is, what was the motivation? Was it just to > > > >> have a more pleasing user-to-user communications experience, or was > > > >> discussion across the network an explicit goal? There's a note in > > > >> talk.c ("Modified to run between hosts by Peter Moore, 8/19/82") > that > > > >> suggests this wasn't the original intent. Who thought up the > > > >> character-at-a-time display mode? > > > >> > > > >> Thanks for any insights. > > > >> > > > >> - Dan C. > > > >> > > > > > > > > -- > > --- > > Larry McVoy Retired to fishing > http://www.mcvoy.com/lm/boat > -------------- next part -------------- An HTML attachment was scrubbed... URL: From crossd at gmail.com Sun Dec 15 06:03:30 2024 From: crossd at gmail.com (Dan Cross) Date: Sat, 14 Dec 2024 15:03:30 -0500 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> Message-ID: On Sat, Dec 14, 2024 at 2:13 PM Marc Rochkind wrote: > [snip] > Is there a DVCS out there that's an alternative to Git? >[snip] A few! Darcs, bazaar, mercurial are examples. Mercurial was fairly popular 10 or 15 years ago, before git fully took over the mindshare. A new VCS that a lot of folks are very excited about is Jujutsu: https://github.com/martinvonz/jj. It's git compatible, and according to their README uses git repositories as the "storage layer". - Dan C. From lars at nocrew.org Sun Dec 15 06:41:13 2024 From: lars at nocrew.org (Lars Brinkhoff) Date: Sat, 14 Dec 2024 20:41:13 +0000 Subject: [TUHS] BSD talk program? In-Reply-To: <7wplluloh1.fsf@junk.nocrew.org> (Lars Brinkhoff's message of "Sat, 14 Dec 2024 07:20:58 +0000") References: <7wplluloh1.fsf@junk.nocrew.org> Message-ID: <7w8qsiknfa.fsf@junk.nocrew.org> > It's TCP port 87 in RFC 1060 "Assigned Numbers", but seems to have > been dropped since. Checking closer, I see it was first defined as "any terminal link" in RFC 770 from 1980. It was updated to "any private terminal link" in 1984. TTYLINK is actually a "Unix port" in RFC 1060. RFC 1340 dropped TTYLINK but retains "any private terminal link", and it's still there in RFC 1700 from 1994, and in the IANA database today. https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=87&page=1 From luther.johnson at makerlisp.com Sun Dec 15 06:54:27 2024 From: luther.johnson at makerlisp.com (Luther Johnson) Date: Sat, 14 Dec 2024 13:54:27 -0700 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> Message-ID: <295396a1-2dc8-871a-3d46-beda6dd1f697@makerlisp.com> "Is there a DVCS out there that's an alternative to Git?" I'm sure the list will offer many options, but I think you might want to look into mentioning either Fossil or Mercurial as alternatives to Git. I have used Mercurial a tiny bit, and I have used Fossil regularly over the last several years. Considering commercial (not open source) software, I think Perforce is another candidate, I have used it before, not lately, but at the time I used it, I think it had a reasonably large customer base. On 12/14/2024 12:13 PM, Marc Rochkind wrote: > Thanks, Larry and Branden. I'll try to work what you're saying into my > paper. > > Larry, I found your post of a few days ago > (https://www.tuhs.org/pipermail/tuhs/2024-December/031188.html) very > illuminating, and I'd like to reference it, which I could do with that > post. But, is there some other, more referenceable article saying the > same thing that I can reference instead? > > Regarding BK vs. Git. Even if BK is superior, can today's developers > use BK? My understanding is that it is no longer supported. > > Is there a DVCS out there that's an alternative to Git? > > The reason why this part of my paper has to be short is that I'm > discussing the influence of SCCS on software engineering, and its > influence on DVCSs is very indirect. My paper isn't a history of VCSs. > > Marc > > On Sat, Dec 14, 2024 at 11:53 AM Larry McVoy > wrote: > > On Sat, Dec 14, 2024 at 12:25:56PM -0600, G. Branden Robinson wrote: > > At 2024-12-14T10:43:47-0700, Marc Rochkind wrote: > > > As I mentioned in another post, I'm writing an invited paper > for an > > > upcoming issue of IEEE Transactions on Software Engineering > that will > > > be a 50-year retrospective of my original 1975 SCCS paper ( > > > mrochkind.com/aup/talks/SCCS-Slideshow.pdf > ). Can some > people here > > > review a couple of paragraphs for accuracy? > > > > I apologize for this not being quite responsive to your query, then. > > > > 1. Tom Lord's Arch (ca. 2001) was at one timely recognized for > > popularizing, or perhaps _introducing_, the notion of > decentralized > > version control to the dot-com era of hard-scrabble FLOSS > developers > > who were building "Web 2.0" and whose startup employers, be they > > flush with cash or not, generally would not spring for a > commercial > > VCS (which might not have been decentralized anyway). > > BitKeeper was 3 years into development by then. Here is the first BK > changeset: > > ChangeSet at 1.1, 1999-03-19 16:11:26-08:00, lm at lm.bitmover.com > > BitKeeper gets stored in BitKeeper. > > > The most salient point here is that Linus Torvalds was not > prescient > > or uniquely perceptive in recognizing the value of a DVCS. > > Couldn't agree more. He, Dave Miller, and Richard Henderson came > to my > house in SF because Dave was threatening to fork Linux because Linus > used no SCM at all. That pushed all the merges to the next level, to > people like Dave. It took me 4-5 hours of explaining how distributed > SCM would work for Linus to get it. My wife was unimpressed with him. > Whatever, at the end of that day, he said "build that and I'll use > it". > And we did, he did, and rate of development doubled (according to the > people who hated me) and was actually about 10x faster (according > to the > people who could do math). > > > 2. "TeamWare and BitKeeper took advantage of the interleaved delta > > algorithm, also known as a weave, to implement an efficient > way to > > represent merged deltas by reference, instead of reproducing > code > > inside the repository. This is a lot more complicated to do with > > reverse deltas, introduced by RCS.*" > > > > I'd a like a second footnote directing me to where I can > understand > > the mathematics supporting this claim. Just out of nerd > interest. > > See if this helps: > > https://www.tuhs.org/pipermail/tuhs/2024-December/031188.html > > You can work out that SCCS/BK are doing what they claim by running > bk annotate on a file that was authored by X but automerged by Y. > The authorship stays the same across the merge, that's not true in > most SCM systems that copy the data from the branch to the trunk. > > Happy to answer any questions. And, yes, git is a ginormous step > backwards. Only one graph, BK had a graph per file so the GCA is > always the correct one, none of this try different ones until you > get one that automerges; since there is only one graph, all you > get are commit comments, you lose the oh-so-valuable-file-comments > that you need when the crap hits the fan; you lose a boat load of > performance, especially in areas like annotate/blame when the repo > gets big, etc. Git sucks, it really does. Linus claims he wrote > it in a week and it shows. > -- > --- > Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat > > > > -- > /My new email address is mrochkind at gmail.com / -------------- next part -------------- An HTML attachment was scrubbed... URL: From d235j.1 at gmail.com Sun Dec 15 07:08:20 2024 From: d235j.1 at gmail.com (David Ryskalczyk) Date: Sat, 14 Dec 2024 16:08:20 -0500 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <295396a1-2dc8-871a-3d46-beda6dd1f697@makerlisp.com> References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <295396a1-2dc8-871a-3d46-beda6dd1f697@makerlisp.com> Message-ID: I've seen a lot of discussion in recent days about Jujutsu. It's a standalone DVCS, but the primary production backend uses Git, making it significantly more practical to use than the other options. https://github.com/martinvonz/jj https://neugierig.org/software/blog/2024/12/jujutsu.html https://ahal.ca/blog/2024/jujutsu-mercurial-haven/ David > On Dec 14, 2024, at 3:54 PM, Luther Johnson wrote: > > "Is there a DVCS out there that's an alternative to Git?" > > I'm sure the list will offer many options, but I think you might want to look into mentioning either Fossil or Mercurial as alternatives to Git. I have used Mercurial a tiny bit, and I have used Fossil regularly over the last several years. Considering commercial (not open source) software, I think Perforce is another candidate, I have used it before, not lately, but at the time I used it, I think it had a reasonably large customer base. > > On 12/14/2024 12:13 PM, Marc Rochkind wrote: >> Thanks, Larry and Branden. I'll try to work what you're saying into my paper. >> >> Larry, I found your post of a few days ago (https://www.tuhs.org/pipermail/tuhs/2024-December/031188.html) very illuminating, and I'd like to reference it, which I could do with that post. But, is there some other, more referenceable article saying the same thing that I can reference instead? >> >> Regarding BK vs. Git. Even if BK is superior, can today's developers use BK? My understanding is that it is no longer supported. >> >> Is there a DVCS out there that's an alternative to Git? >> >> The reason why this part of my paper has to be short is that I'm discussing the influence of SCCS on software engineering, and its influence on DVCSs is very indirect. My paper isn't a history of VCSs. >> >> Marc >> >> On Sat, Dec 14, 2024 at 11:53 AM Larry McVoy > wrote: >>> On Sat, Dec 14, 2024 at 12:25:56PM -0600, G. Branden Robinson wrote: >>> > At 2024-12-14T10:43:47-0700, Marc Rochkind wrote: >>> > > As I mentioned in another post, I'm writing an invited paper for an >>> > > upcoming issue of IEEE Transactions on Software Engineering that will >>> > > be a 50-year retrospective of my original 1975 SCCS paper ( >>> > > mrochkind.com/aup/talks/SCCS-Slideshow.pdf ). Can some people here >>> > > review a couple of paragraphs for accuracy? >>> > >>> > I apologize for this not being quite responsive to your query, then. >>> > >>> > 1. Tom Lord's Arch (ca. 2001) was at one timely recognized for >>> > popularizing, or perhaps _introducing_, the notion of decentralized >>> > version control to the dot-com era of hard-scrabble FLOSS developers >>> > who were building "Web 2.0" and whose startup employers, be they >>> > flush with cash or not, generally would not spring for a commercial >>> > VCS (which might not have been decentralized anyway). >>> >>> BitKeeper was 3 years into development by then. Here is the first BK >>> changeset: >>> >>> ChangeSet at 1.1 , 1999-03-19 16:11:26-08:00, lm at lm.bitmover.com >>> BitKeeper gets stored in BitKeeper. >>> >>> > The most salient point here is that Linus Torvalds was not prescient >>> > or uniquely perceptive in recognizing the value of a DVCS. >>> >>> Couldn't agree more. He, Dave Miller, and Richard Henderson came to my >>> house in SF because Dave was threatening to fork Linux because Linus >>> used no SCM at all. That pushed all the merges to the next level, to >>> people like Dave. It took me 4-5 hours of explaining how distributed >>> SCM would work for Linus to get it. My wife was unimpressed with him. >>> Whatever, at the end of that day, he said "build that and I'll use it". >>> And we did, he did, and rate of development doubled (according to the >>> people who hated me) and was actually about 10x faster (according to the >>> people who could do math). >>> >>> > 2. "TeamWare and BitKeeper took advantage of the interleaved delta >>> > algorithm, also known as a weave, to implement an efficient way to >>> > represent merged deltas by reference, instead of reproducing code >>> > inside the repository. This is a lot more complicated to do with >>> > reverse deltas, introduced by RCS.*" >>> > >>> > I'd a like a second footnote directing me to where I can understand >>> > the mathematics supporting this claim. Just out of nerd interest. >>> >>> See if this helps: >>> >>> https://www.tuhs.org/pipermail/tuhs/2024-December/031188.html >>> >>> You can work out that SCCS/BK are doing what they claim by running >>> bk annotate on a file that was authored by X but automerged by Y. >>> The authorship stays the same across the merge, that's not true in >>> most SCM systems that copy the data from the branch to the trunk. >>> >>> Happy to answer any questions. And, yes, git is a ginormous step >>> backwards. Only one graph, BK had a graph per file so the GCA is >>> always the correct one, none of this try different ones until you >>> get one that automerges; since there is only one graph, all you >>> get are commit comments, you lose the oh-so-valuable-file-comments >>> that you need when the crap hits the fan; you lose a boat load of >>> performance, especially in areas like annotate/blame when the repo >>> gets big, etc. Git sucks, it really does. Linus claims he wrote >>> it in a week and it shows. >>> -- >>> --- >>> Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat >> >> >> >> -- >> My new email address is mrochkind at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at oneiros.de Sun Dec 15 07:18:09 2024 From: martin at oneiros.de (=?UTF-8?Q?Martin_Schr=C3=B6der?=) Date: Sat, 14 Dec 2024 22:18:09 +0100 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> Message-ID: Am Sa., 14. Dez. 2024 um 20:14 Uhr schrieb Marc Rochkind : > Is there a DVCS out there that's an alternative to Git? https://en.wikipedia.org/wiki/Comparison_of_version-control_software might be a start. And of course needs updates. Best Martin From woods at robohack.ca Sun Dec 15 10:53:01 2024 From: woods at robohack.ca (Greg A. Woods) Date: Sat, 14 Dec 2024 16:53:01 -0800 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: Hi Marc, This is more of an aside, but as a long time and ongoing SCCS user I've wondered about the vc(1) command included in the SCCS tool suite. So I was hoping you (or anyone else reading) might be able to shed some light on its origins and maybe give, or point to, some examples of how it was intended to be used, or indeed how it ended up being used. When I first encountered it, which would have been in the mid-1980s, I thought it might be useful to help build a proper configuration management system on top of or alongside SCCS, but I've never managed to hack anything like that together myself, nor was I ever able to find any examples of its use that might help me. Most people I asked about vc(1) were dismissive and suggested it was archaic and useless, or as with this time with a tangentially related question where all I got was silence: https://mirrors.nycbug.org/pub/The_Unix_Archive/Unix_Usenet/comp.unix.questions/1988-August/012896.html -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From tytso at mit.edu Sun Dec 15 13:34:42 2024 From: tytso at mit.edu (Theodore Ts'o) Date: Sat, 14 Dec 2024 22:34:42 -0500 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> Message-ID: <20241215033442.GD2472262@mit.edu> On Sat, Dec 14, 2024 at 03:03:30PM -0500, Dan Cross wrote: > > A few! Darcs, bazaar, mercurial are examples. Mercurial was fairly > popular 10 or 15 years ago, before git fully took over the mindshare. There is no question that there are other DCVS's which are more sophisticated than git. But one could argue that there are other OS's which are more "sophisticated", and that was their argument for why they were better than Unix. For example, I've had Multicians tell me about super-sophistcated things which Unix or any more modern Unix descendants *still* don't do, which they will claim is a reason why Multics is "superior" to Unix. But yet, there's no question that Multics was a commercial failure, and that Unix's simplicity and ease of porting to other architectures besides the DPS-8/M was one of the reasons why Unix succeeded and Multics has been consigned to the dustbin of history. So I find it interesting how one the one hand, we will extol the virtues of Unix's simplicity in terms of using byte streams for files, as oposed to the horrendous record-based files used by Mainframes and VMS, And yet we can *also* say that Git is stupid because it doesn't track renames, while Bitkeeper does, and therefore it is better than the simple blob-based model which is git. One could make the argumet that "git was lucky; it was in the right place at the right time, and managed to get mindshare", but I suspect there are other people who might say the same thing about Unix and it getting mindshare more out of luck than the fundametal superiority of its technical architecture. And I think there are many on this list who would contest mightily that Unix was "just lucky". Food for thought.... - Ted From flexibeast at gmail.com Sun Dec 15 14:15:57 2024 From: flexibeast at gmail.com (Alexis) Date: Sun, 15 Dec 2024 15:15:57 +1100 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241215033442.GD2472262@mit.edu> (Theodore Ts'o's message of "Sat, 14 Dec 2024 22:34:42 -0500") References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> Message-ID: <8734ipmvia.fsf@gmail.com> "Theodore Ts'o" writes: > [T]here's no question that Multics was a commercial failure, and > that > Unix's simplicity and ease of porting to other architectures > besides > the DPS-8/M was one of the reasons why Unix succeeded and > Multics has > been consigned to the dustbin of history. > > ... > > One could make the argumet that "git was lucky; it was in the > right > place at the right time, and managed to get mindshare", but I > suspect > there are other people who might say the same thing about Unix > and it > getting mindshare more out of luck than the fundametal > superiority of > its technical architecture. > > And I think there are many on this list who would contest > mightily > that Unix was "just lucky". Gabriel's "Worse Is Better" essay is coming irresistibly to mind .... i'd certainly be interested in this list's opinion of that piece, 30+ years later. Alexis. From lars at nocrew.org Sun Dec 15 17:22:29 2024 From: lars at nocrew.org (Lars Brinkhoff) Date: Sun, 15 Dec 2024 07:22:29 +0000 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241215033442.GD2472262@mit.edu> (Theodore Ts'o's message of "Sat, 14 Dec 2024 22:34:42 -0500") References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> Message-ID: <7wzfkxjtqi.fsf@junk.nocrew.org> Theodore Ts'o wrote: > But yet, there's no question that Multics was a commercial failure By which metric? Honest question. Multics seems to have been in business around 1975-2000, but I don't know if it was in the read or in the black. From sjenkin at canb.auug.org.au Sun Dec 15 18:16:49 2024 From: sjenkin at canb.auug.org.au (sjenkin at canb.auug.org.au) Date: Sun, 15 Dec 2024 19:16:49 +1100 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: Greg, I was never much of a developer, have written little code in last 25yrs, more download, compile and use now :) And in that light, used a number of VCS, open source & not, over the years. Not a ‘power user’ :( but not unpracticed. Reading the SCCS docs, ‘what’ and ‘vc’ seem related, placing / looking for version related strings in code at a minimum. ‘what’ obviously is similar RCS’ ‘ident’, has its own SCCS id list. I can’t think of an equivalent of ‘vc’ in RCS - doing something Configuration related. I’ve done config management & conversion / port to new platforms on a few modest systems (1M lines of code) and can see what you’re getting at with ‘vc’ and its ability to take any file, SCCS or not, and rewrite them using AWK-like functionality. There’s a nomenclature quirk problem in the man pages I had to get past: ‘control character’ is used to mean the (ascii) delimiter for matched/substituted strings. With more a comms background, I first read it as low-value ASCII chars :) I stumbled on Tony Finch’s blog, author of ‘unifdef’, and how he displays the version string in that program. Inferences: - the people who wrote SCCS used ‘vc’, or people very close to them needed it, had it written for them. - it’s NOT trying to replicate Identifiers, as RCS ident does. This is quite different functionality. - ‘vc’ works on non-SCCS files, so it can work on make files, config, release & distribution scripts… - the awk-like syntax, delimiter + keyword substitution suggests you’d not apply it to code - too easy to make a huge mess. As you suggested, have a Configuration SCCS Repo storing all the components / version-id of all files for a Release to a set of target platforms, and some scripts + ‘vc’ to reliably generate builds making bit-identical objects for a release. First part of any good production acceptance test process would be ‘build it all from source’ - and check all created objects were bit-identical to Release Candidate. I’ve heard people complain with ‘git’ that for some projects the compile time/date string is included - making invariant object builds impossible… A retrograde step IMHO. This would make sense if you were Bell Labs / USL around 1984 building source for many hardware targets, many O/S platforms. They’d have automated the build/release process and (hopefully) insisted on bit-identical objects. I’m not sure how many people understand Configuration Management - its subtle, even supporting two versions of the same system & tools. I did enough to know I didn’t know much about it :) cheers steve [ Notes at end ] > On 15 Dec 2024, at 11:53, Greg A. Woods wrote: > > Hi Marc, > > This is more of an aside, but as a long time and ongoing SCCS user I've > wondered about the vc(1) command included in the SCCS tool suite. > > So I was hoping you (or anyone else reading) might be able to shed some > light on its origins and maybe give, or point to, some examples of how > it was intended to be used, or indeed how it ended up being used. > > When I first encountered it, which would have been in the mid-1980s, I > thought it might be useful to help build a proper configuration > management system on top of or alongside SCCS, but I've never managed to > hack anything like that together myself, nor was I ever able to find any > examples of its use that might help me. > > Most people I asked about vc(1) were dismissive and suggested it was > archaic and useless, or as with this time with a tangentially related > question where all I got was silence: > > https://mirrors.nycbug.org/pub/The_Unix_Archive/Unix_Usenet/comp.unix.questions/1988-August/012896.html > > > -- > Greg A. Woods > > Kelowna, BC +1 250 762-7675 RoboHack > Planix, Inc. Avoncote Farms =============== Notes. Inserting SCCS ID keywords in a file 3.2.1.2 Inserting SCCS ID Keywords ident AIX 4.3 what Command AIX 4.3 List of Additional SCCS Commands vc Substitutes assigned values for identification keywords. vc -- version control (SCCS) The vc command is awk-like tool used for version control of sets of files. While it is distributed as part of the SCCS package, it does not require the files it operates on to be under SCCS control. =============== Tony Finch 2024-05-13 – Unix version control lore: what, ident [ on my mac ] [ both what & ident report no tags in /usr/bin/unifdef ] steve$ unifdef -V Version: unifdef-2.5.6.21f1388 FreeBSD: src/usr.bin/unifdef/unifdef.c,v 1.31 2011/01/21 18:10:11 fanf Exp Author: Tony Finch (dot at dotat.at) URL: http://dotat.at/prog/unifdef Each line is prefixed with an SCCS magic marker @(#) so that what can find it, and wrapped in an RCS-style $Keyword$so that ident can find it. There’s a fairly trivial version() function that spits out the copyright[] string when you run unifdef -V. embedding versions from git My projects have various horrible build scripts for embedding the version number from git. The basic idea is, • use an annotated or signed tag to mark a release, i.e. git tag -a or git tag -s • use git describe to get a version string that includes an extra number counting commits since the last release • maybe use git show --pretty=format:%ai -s HEAD to get a release date • stuff the outputs from git into the $Version$ and $Date$ RCS keywords I enjoy keeping this old feature working, even though it isn’t very useful if no-one knows about it! Maybe if I blog about it, it’ll become more widespread? =============== -- Steve Jenkin, IT Systems and Design 0412 786 915 (+61 412 786 915) PO Box 38, Kippax ACT 2615, AUSTRALIA mailto:sjenkin at canb.auug.org.au http://members.tip.net.au/~sjenkin From sjenkin at canb.auug.org.au Sun Dec 15 18:20:56 2024 From: sjenkin at canb.auug.org.au (sjenkin at canb.auug.org.au) Date: Sun, 15 Dec 2024 19:20:56 +1100 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: <3B97F5D3-7F18-487C-8EBF-476373440ED6@canb.auug.org.au> Apologies for SPAMing the list - was meant to be a personal note to Greg. My mail program bested me :( > On 15 Dec 2024, at 19:16, sjenkin at canb.auug.org.au wrote: > > Greg, > > I was never much of a developer, have written little code in last 25yrs, > more download, compile and use now :) > -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.branden.robinson at gmail.com Sun Dec 15 19:34:47 2024 From: g.branden.robinson at gmail.com (G. Branden Robinson) Date: Sun, 15 Dec 2024 03:34:47 -0600 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> Message-ID: <20241215093447.fdbbv3vb6czxgmji@illithid> [replying to Marc's message because Larry's is not in my inbox] At 2024-12-14T12:13:40-0700, Marc Rochkind wrote: > On Sat, Dec 14, 2024 at 11:53 AM Larry McVoy wrote: > > On Sat, Dec 14, 2024 at 12:25:56PM -0600, G. Branden Robinson wrote: > > > At 2024-12-14T10:43:47-0700, Marc Rochkind wrote: > > > 1. Tom Lord's Arch (ca. 2001) was at one timely recognized for > > > popularizing, or perhaps _introducing_, the notion of > > > decentralized version control to the dot-com era of hard-scrabble > > > FLOSS developers who were building "Web 2.0" and whose startup > > > employers, be they flush with cash or not, generally would not > > > spring for a commercial VCS (which might not have been > > > decentralized anyway). > > > > BitKeeper was 3 years into development by then. I get that you're attentive to the matter of priority here, but I wrote my paragraph carefully--although I see not quite carefully enough. Arch popularized or introduced the notion of decentralized version control to hard-scrabble FLOSS developers in the dot-com era whose employers would not spend money on a commercial VCS. I recollect now that BK was available under a free-to-use license--as long as one didn't get too curious, like Andrew Tridgell, or wish to remain at liberty to participate in FLOSS projects that were themselves VCSes. Such a restriction was unacceptable to engineers in my circle (Debian people, and Debian-adjacent people through my workplace). From our perspective, BitKeeper was about as affordable and as appealing as ClearCase. And at my workplace, management wasn't interested in paying money to cram down a VCS that the engineers would grumble about using. We muddled through, mostly with Subversion, as I recall. We didn't have to face the scale of coordination challenge that the Linux kernel did. > > > 2. "TeamWare and BitKeeper took advantage of the interleaved > > > delta algorithm, also known as a weave, to implement an > > > efficient way to represent merged deltas by reference, instead > > > of reproducing code inside the repository. This is a lot more > > > complicated to do with reverse deltas, introduced by RCS.*" > > > > > > I'd a like a second footnote directing me to where I can > > > understand the mathematics supporting this claim. Just out of > > > nerd interest. > > > > See if this helps: > > > > https://www.tuhs.org/pipermail/tuhs/2024-December/031188.html I saw that, and it sheds some light, but it doesn't rise to the level of a theorem. If I were a mathematician, I might know what analytical discipline to bring to bear on my question to you, but the best I can do is to mumble about how it looks like you need both graph theory and set theory for this. And I lack the category theory or other means to understand whether and how those two might compose. And I don't know that category theory would even help; it simply seems to be the device most often employed to, uh, uncover higher-level isomorphisms in problem domains. (That's where I duck a tomato hurled from the audience.) A good mathematical expositor could, I add, employ these tools without leaning too hard on the formalisms, and produce a writeup that is broadly accessible. We used to have USENIX for this sort of thing... > > You can work out that SCCS/BK are doing what they claim by running > > bk annotate on a file that was authored by X but automerged by Y. > > The authorship stays the same across the merge, that's not true in > > most SCM systems that copy the data from the branch to the trunk. I dimly recall that the "merge problem" was one of the shoals upon which Subversion, if not quite ran aground upon, found itself adjacently becalmed. > > Happy to answer any questions. I've noted your enthusiasm for the weave and BK's amplification of the concept. What I think you need is, as noted, a mathematical expositor who can express the novelty of Rochkind's and your contributions in terms that professionals who have little contact with the problems of "source code configuration management" (an alternative nomenclature for "version control" I've encountered) can comprehend. You've tried popularizing to the masses. My conclusion is that, at best, they stare slackly at you and say, "Git does that. I use Git.". To get your innovation more broadly recognized, you may therefore have to take your case to the ivory tower. Despite the university training that many (most?) software engineers receive, and the many tools that other engineering disciplines and applied mathematics have to offer our field, there is little cross- pollination. Relatively few programmers actually read Dijkstra or Hoare. You're doing well if you can get them to read Brooks. Too often, in my view, the only value "theoretical computer science"[1] has is as a fountainhead of buzzwords one can spray on a slide deck for one's manager to pitch to a promotion committee. I had to go to work at a research lab to find an environment where people comfortably moved back and forth between these domains. Naturally, its budget got savaged. The grandparent organization wanted "an AI story". Regards, Branden [1] I strongly dislike that term because of what it implies when the modifier is deleted: if you have a science without a theoretical framework, what you have is not a science. I suspect it is preferred by math-phobes and managers who won't look past an impending deadline. ("Who cares if it won't scale? That's a problem for NEXT quarter. Get it into production.") -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From g.branden.robinson at gmail.com Sun Dec 15 19:43:58 2024 From: g.branden.robinson at gmail.com (G. Branden Robinson) Date: Sun, 15 Dec 2024 03:43:58 -0600 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <7wzfkxjtqi.fsf@junk.nocrew.org> References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> <7wzfkxjtqi.fsf@junk.nocrew.org> Message-ID: <20241215094358.ilgv62mui7tyxhdo@illithid> At 2024-12-15T07:22:29+0000, Lars Brinkhoff wrote: > Theodore Ts'o wrote: > > But yet, there's no question that Multics was a commercial failure > > By which metric? Honest question. Multics seems to have been in > business around 1975-2000, but I don't know if it was in the read or > in the black. I note that that's longer than MS-DOS was a commercial product. People seem mighty quick to use words like "irrelevant" or "failure" as a substitute for reasoned argument. Maybe I would, in fact, hate using Multics. But I can't forget that well after Unix had fledged, its developers at CSRC found it necessary and/or desirable to borrow back a Multics concept: they named it mmap(). Not having been concieved as desirable from the start, it was grafted on, with negative consequences. The archives of this list feature multiple war stories from Larry McVoy about how, as I recollect, unifying the buffer cache was a dragon that bedeviled every version of Unix until SunOS 4 finally slayed it. (Have I got that right?) And promptly got chucked overboard for a System V kernel because that's how glorious the software industry is. Regards, Branden -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From robpike at gmail.com Sun Dec 15 20:05:37 2024 From: robpike at gmail.com (Rob Pike) Date: Sun, 15 Dec 2024 21:05:37 +1100 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241215094358.ilgv62mui7tyxhdo@illithid> References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> <7wzfkxjtqi.fsf@junk.nocrew.org> <20241215094358.ilgv62mui7tyxhdo@illithid> Message-ID: Google ran on Perforce in the early days, then on an internal reimplementation that could scale better (long story). I believe that's still the case, as git continues to be desired by the users but unworkable as the core repo at the required scale. Clever hackery allows git to feel like it's being used, but it's all above what was at least at one time Perforce. My info could well be out of date, though. -rob On Sun, Dec 15, 2024 at 8:44 PM G. Branden Robinson < g.branden.robinson at gmail.com> wrote: > At 2024-12-15T07:22:29+0000, Lars Brinkhoff wrote: > > Theodore Ts'o wrote: > > > But yet, there's no question that Multics was a commercial failure > > > > By which metric? Honest question. Multics seems to have been in > > business around 1975-2000, but I don't know if it was in the read or > > in the black. > > I note that that's longer than MS-DOS was a commercial product. > > People seem mighty quick to use words like "irrelevant" or "failure" as > a substitute for reasoned argument. > > Maybe I would, in fact, hate using Multics. But I can't forget that > well after Unix had fledged, its developers at CSRC found it necessary > and/or desirable to borrow back a Multics concept: they named it mmap(). > > Not having been concieved as desirable from the start, it was grafted > on, with negative consequences. The archives of this list feature > multiple war stories from Larry McVoy about how, as I recollect, > unifying the buffer cache was a dragon that bedeviled every version of > Unix until SunOS 4 finally slayed it. (Have I got that right?) > > And promptly got chucked overboard for a System V kernel because that's > how glorious the software industry is. > > Regards, > Branden > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lars at nocrew.org Sun Dec 15 22:44:06 2024 From: lars at nocrew.org (Lars Brinkhoff) Date: Sun, 15 Dec 2024 12:44:06 +0000 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241215094358.ilgv62mui7tyxhdo@illithid> (G. Branden Robinson's message of "Sun, 15 Dec 2024 03:43:58 -0600") References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> <7wzfkxjtqi.fsf@junk.nocrew.org> <20241215094358.ilgv62mui7tyxhdo@illithid> Message-ID: <7wseqpjeuh.fsf@junk.nocrew.org> G. Branden Robinson wrote: > well after Unix had fledged, its developers at CSRC found it necessary > and/or desirable to borrow back a Multics concept: they named it mmap(). It has been argued the borrowing was from TENEX. TENEX in turn would have been influenced by Multics and the SDS 940 timesharing system. From lm at mcvoy.com Mon Dec 16 00:31:00 2024 From: lm at mcvoy.com (Larry McVoy) Date: Sun, 15 Dec 2024 06:31:00 -0800 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241215093447.fdbbv3vb6czxgmji@illithid> References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215093447.fdbbv3vb6czxgmji@illithid> Message-ID: <20241215143100.GC9825@mcvoy.com> On Sun, Dec 15, 2024 at 03:34:47AM -0600, G. Branden Robinson wrote: > > > > 2. "TeamWare and BitKeeper took advantage of the interleaved > > > > delta algorithm, also known as a weave, to implement an > > > > efficient way to represent merged deltas by reference, instead > > > > of reproducing code inside the repository. This is a lot more > > > > complicated to do with reverse deltas, introduced by RCS.*" > > > > > > > > I'd a like a second footnote directing me to where I can > > > > understand the mathematics supporting this claim. Just out of > > > > nerd interest. > > > > > > See if this helps: > > > > > > https://www.tuhs.org/pipermail/tuhs/2024-December/031188.html > > I saw that, and it sheds some light, but it doesn't rise to the level of > a theorem. Try rereading it a few times. And work through the various versions of the history file I wrote out. Everything you need to understand is right there. I get that it isn't easy, but it isn't that hard either. > If I were a mathematician, I might know what analytical discipline to > bring to bear on my question to you, but the best I can do is to mumble > about how it looks like you need both graph theory and set theory for > this. You need to be able to visualize a graph, yes. And Rick would agree you need some education to _really_ understand everything. I'd argue you need very little to get the basics. Remember, the claim is that an automerge is a set thing only, no changes to the weave. I demonstrated that. > A good mathematical expositor could, I add, employ these tools without > leaning too hard on the formalisms, and produce a writeup that is > broadly accessible. > > We used to have USENIX for this sort of thing... Indeed. And if USENIX were still a thing, I might write that paper. I'm not particularly motivated. I participated in a 3 day SCM something a while back, put on by google and facebook. I spent 3 days listening to their problems, and on most of them, I said "yeah, we solved that, here is what we did". And then watched while they ignored everything. So I'm not hopeful that people will get it. If I were younger, I might write the paper anyway but I'm not. I tried, BK is open source, you can go read the code (I know of one guy who did and came back claiming it was the most pleasant C source base he had ever seen, but that's it. Noone else has said boo). > I've noted your enthusiasm for the weave and BK's amplification of the > concept. What I think you need is, as noted, a mathematical expositor > who can express the novelty of Rochkind's and your contributions in > terms that professionals who have little contact with the problems of > "source code configuration management" (an alternative nomenclature for > "version control" I've encountered) can comprehend. You've tried > popularizing to the masses. My conclusion is that, at best, they stare > slackly at you and say, "Git does that. I use Git.". To get your > innovation more broadly recognized, you may therefore have to take your > case to the ivory tower. You're almost there. What they say is "I use Github". Github has dumbed down DVCS to the point they aren't much different than CVS. When I realized that years ago, I retired. My belief is BK is sort of like betamax, it's better but VHS won. It is what it is. From tuhs at tuhs.org Mon Dec 16 02:33:13 2024 From: tuhs at tuhs.org (Bakul Shah via TUHS) Date: Sun, 15 Dec 2024 08:33:13 -0800 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241215143100.GC9825@mcvoy.com> References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215093447.fdbbv3vb6czxgmji@illithid> <20241215143100.GC9825@mcvoy.com> Message-ID: <76AF4F63-A044-40CA-8D4E-741738D70DEA@iitbombay.org> On Dec 15, 2024, at 6:31 AM, Larry McVoy wrote: > > Indeed. And if USENIX were still a thing, I might write that paper. I'm > not particularly motivated. I participated in a 3 day SCM something a > while back, put on by google and facebook. I spent 3 days listening to > their problems, and on most of them, I said "yeah, we solved that, here > is what we did". And then watched while they ignored everything. > > So I'm not hopeful that people will get it. If I were younger, I might > write the paper anyway but I'm not. A well written paper (as opposed to random discussions in a meeting with multiple interruptions) would definitely help people get it. I certainly would read it, in preference to 127K+ LoC in bitkeeper/src! >> I've noted your enthusiasm for the weave and BK's amplification of the >> concept. What I think you need is, as noted, a mathematical expositor >> who can express the novelty of Rochkind's and your contributions in >> terms that professionals who have little contact with the problems of >> "source code configuration management" (an alternative nomenclature for >> "version control" I've encountered) can comprehend. You've tried >> popularizing to the masses. My conclusion is that, at best, they stare >> slackly at you and say, "Git does that. I use Git.". To get your >> innovation more broadly recognized, you may therefore have to take your >> case to the ivory tower. > > You're almost there. What they say is "I use Github". Github has > dumbed down DVCS to the point they aren't much different than CVS. > When I realized that years ago, I retired. My belief is BK is sort of > like betamax, it's better but VHS won. It is what it is. Purely as a thought experiment & with the benefit of hindsight: What would you do differently if you were to reimplement bk? What were some things you wanted to do but didn't get around to? Include any blue sky ideas! Include any simplifying ideas! Given that there are newer than git efforts like jujitsu and pijul etc., git won't be the last word. Though it hasn't reached the VHS stage of obsolescence! From mrochkind at gmail.com Mon Dec 16 03:49:36 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Sun, 15 Dec 2024 10:49:36 -0700 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: On Sat, Dec 14, 2024 at 6:43 PM Greg A. Woods wrote: > Hi Marc, > > This is more of an aside, but as a long time and ongoing SCCS user I've > wondered about the vc(1) command included in the SCCS tool suite. > > So I was hoping you (or anyone else reading) might be able to shed some > light on its origins and maybe give, or point to, some examples of how > it was intended to be used, or indeed how it ended up being used. > I had completely forgotten about the vc command until I read this post. Vc was a language-independent macro processor mostly concerned with excluding or including lines of text based on the evaluation of logical expressions. Even though "vc" stands for "version control," it has nothing to do with version control as that phrase is used nowadays. In the early 1970s where I worked at Bell Labs, we (or at least I) used the term "release" to refer to the different forms of a file as development proceeded, and "version" to refer to ongoing variants of the file. For example, there might be a version of some program for Southwestern Bell and a different version for New York Telephone. Those two versions were contained in the same identical sets of source files. I just looked again at my 1975 paper and I don't think the word "version" appears anywhere. The terms "release" and "level" appear. I designed vc, but it was implemented by a college student who was with us for the Summer. I credit a woman named Sabrina Feczko in my paper, and I think it was her. As I mentioned, vc was totally standalone so Sabrina didn't have to concern herself with all the squirrely code in the main part of SCCS. Vc disappeared at some point. I don't think it was ever used, as it was intended for mainframe programs that were being developed (but not compiled) on PWB/UNIX. For UNIX development, C had its own macro processor. Incidentally, UNIX had a different language-independent macro processor called m6. I think it was created by Doug McIlroy and Andy Hall, and, as I recall, that was even before UNIX became widely used. Hall's M6 implementation was in Fortran, possibly initially for the GE/Honeywell machines running at Murray Hill. Even before I'd heard of UNIX and before I did SCCS, I got the source from Andy and compiled M6 on our IBM mainframe. I'm sure Doug can provide a lot more information about M6. I recall once talking to Andy about a minor bug, and he said that he knew about it, had talked to Doug about it, and Doug said let it go. (I was just starting out at Holmdel and had never met Doug, Andy, or anybody else at Murray Hill. I didn't know it, but Ken and Dennis were writing the first version of UNIX around that time.) Marc -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrochkind at gmail.com Mon Dec 16 04:03:36 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Sun, 15 Dec 2024 11:03:36 -0700 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: On Sun, Dec 15, 2024 at 10:49 AM Marc Rochkind wrote: > > Incidentally, UNIX had a different language-independent macro processor > called m6. I think it was created by Doug McIlroy and Andy Hall, and, as I > recall, that was even before UNIX became widely used. Hall's M6 > implementation was in Fortran, possibly initially for the GE/Honeywell > machines running at Murray Hill. Even before I'd heard of UNIX and before I > did SCCS, I got the source from Andy and compiled M6 on our IBM mainframe. > I'm sure Doug can provide a lot more information about M6. I recall once > talking to Andy about a minor bug, and he said that he knew about it, had > talked to Doug about it, and Doug said let it go. (I was just starting out > at Holmdel and had never met Doug, Andy, or anybody else at Murray Hill. I > didn't know it, but Ken and Dennis were writing the first version of UNIX > around that time.) > Correction: The version of M6 for UNIX was called M4. Maybe because it was only 2/3 as complete? *Doug?* Marc -------------- next part -------------- An HTML attachment was scrubbed... URL: From lm at mcvoy.com Mon Dec 16 04:04:37 2024 From: lm at mcvoy.com (Larry McVoy) Date: Sun, 15 Dec 2024 10:04:37 -0800 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <76AF4F63-A044-40CA-8D4E-741738D70DEA@iitbombay.org> References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215093447.fdbbv3vb6czxgmji@illithid> <20241215143100.GC9825@mcvoy.com> <76AF4F63-A044-40CA-8D4E-741738D70DEA@iitbombay.org> Message-ID: <20241215180437.GE9825@mcvoy.com> On Sun, Dec 15, 2024 at 08:33:13AM -0800, Bakul Shah wrote: > Purely as a thought experiment & with the benefit of hindsight: > What would you do differently if you were to reimplement bk? > What were some things you wanted to do but didn't get around to? > Include any blue sky ideas! Include any simplifying ideas! I'd finish nested repositories. We got them mostly right but we never finished support for renaming a sub repository (what git calls submodules). By "right" I mean you couldn't really tell the difference between a single giant repository and a nested collection of repositories. I'd try to come up with some sort of named branches support. We had a design but never got to it. It's pretty limiting in BK that the only way to have branches is in a bunch of clones. If your repo is big enough, the time/space to rebuild things can get to be too much. As part of the named branches work, we came up with a model that would let you have a DAG but have straight line history as the visible part, you can increase verbosity to get back to the full DAG if you want. The idea was to have "integration trees" where the straight line history is the tip of each push. Lots of people use git rebase to get straight line history and I hate it. You lose lots of useful information but apparently I'm the only guy who cares. > Given that there are newer than git efforts like jujitsu and pijul > etc., git won't be the last word. Though it hasn't reached the VHS > stage of obsolescence! The problem is they are still using git under the covers. That's just a painful step backwards in my mind. -- --- Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From arnold at skeeve.com Mon Dec 16 04:57:38 2024 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sun, 15 Dec 2024 11:57:38 -0700 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: <202412151857.4BFIvcoQ1828802@freefriends.org> > Correction: The version of M6 for UNIX was called M4. Maybe because it was > only 2/3 as complete? IIRC the V6 manual mentions m6, and m4 shows up in V7. But it may have been v5 -> v6 where that happened. If m6 was Fortran and m4 C, then that would (sort of) explain the change. > *Doug?* Yes, exactly, Doug? Thanks, Arnold From tuhs at tuhs.org Mon Dec 16 05:02:03 2024 From: tuhs at tuhs.org (Chet Ramey via TUHS) Date: Sun, 15 Dec 2024 14:02:03 -0500 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241215094358.ilgv62mui7tyxhdo@illithid> References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> <7wzfkxjtqi.fsf@junk.nocrew.org> <20241215094358.ilgv62mui7tyxhdo@illithid> Message-ID: <5e69ed7b-b2be-442a-9b49-6884457084f7@case.edu> On 12/15/24 4:43 AM, G. Branden Robinson wrote: > At 2024-12-15T07:22:29+0000, Lars Brinkhoff wrote: > Maybe I would, in fact, hate using Multics. But I can't forget that > well after Unix had fledged, its developers at CSRC found it necessary > and/or desirable to borrow back a Multics concept: they named it mmap(). I think it was primarily inspired by TENEX/TOPS-20, and it was the CSRG who proposed it. The concepts are described in CSRG TR/4 and mmap() ended up described in the 4.2BSD manual, but not implemented. I don't think Reiser's VM system at the Labs included mmap(), but I think it did have something similar. Of course, that never really saw the light of day. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature.asc Type: application/pgp-signature Size: 203 bytes Desc: OpenPGP digital signature URL: From luther.johnson at makerlisp.com Mon Dec 16 05:12:11 2024 From: luther.johnson at makerlisp.com (Luther Johnson) Date: Sun, 15 Dec 2024 12:12:11 -0700 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <5e69ed7b-b2be-442a-9b49-6884457084f7@case.edu> References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> <7wzfkxjtqi.fsf@junk.nocrew.org> <20241215094358.ilgv62mui7tyxhdo@illithid> <5e69ed7b-b2be-442a-9b49-6884457084f7@case.edu> Message-ID: <08a76bf2-5785-d293-7d84-ef17f2d11263@makerlisp.com> I don't know where mmap showed up in vanilla BSD distributions, but it was in SunOS 4.0, I used it. To the other points, the Multics "one-level-store" idea was very powerful, and was an important influence in Apollo Domain/OS, and it was probably there in Prime Computer's OS too. On 12/15/2024 12:02 PM, Chet Ramey via TUHS wrote: > On 12/15/24 4:43 AM, G. Branden Robinson wrote: >> At 2024-12-15T07:22:29+0000, Lars Brinkhoff wrote: > >> Maybe I would, in fact, hate using Multics. But I can't forget that >> well after Unix had fledged, its developers at CSRC found it necessary >> and/or desirable to borrow back a Multics concept: they named it mmap(). > > I think it was primarily inspired by TENEX/TOPS-20, and it was the CSRG > who proposed it. The concepts are described in CSRG TR/4 and mmap() > ended up described in the 4.2BSD manual, but not implemented. > > I don't think Reiser's VM system at the Labs included mmap(), but I think > it did have something similar. Of course, that never really saw the light > of day. > From johnl at taugh.com Mon Dec 16 06:09:20 2024 From: johnl at taugh.com (John Levine) Date: 15 Dec 2024 15:09:20 -0500 Subject: [TUHS] M macros, wasRe: SCCS In-Reply-To: References: Message-ID: <20241215200920.9D746AE68463@ary.qy> >On Sun, Dec 15, 2024 at 10:49 AM Marc Rochkind wrote: >> Incidentally, UNIX had a different language-independent macro processor >> called m6. ... >Correction: The version of M6 for UNIX was called M4. Maybe because it was >only 2/3 as complete? The Wikipedia article on macroprocessors says that M6 was written in the 1960s by McIlroy, Morris, and Hall, based on GPM and Trac, written in Fortran and ported to v2 Unix. M4 was written in the 1970s by Kernighan and Ritchie in C and is still around, notably as impenetrable magic in GNU autoconfig and sendmail config files. It looks a lot like GPM. As an old Trac user I'd be interested to hear what M6 was like. The Wikipedia article has a footnote pointing to a book by A J Cole, but I found a copy of the book and it says nothing about M6 and has only a passing reference to McIlroy. R's, John From johnl at taugh.com Mon Dec 16 06:12:25 2024 From: johnl at taugh.com (John Levine) Date: 15 Dec 2024 15:12:25 -0500 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <7wseqpjeuh.fsf@junk.nocrew.org> References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241215094358.ilgv62mui7tyxhdo@illithid> <7wzfkxjtqi.fsf@junk.nocrew.org> <20241215033442.GD2472262@mit.edu> <20241214185319.GQ11590@mcvoy.com> <7wseqpjeuh.fsf@junk.nocrew.org> Message-ID: <20241215201225.AFFFFAE684E6@ary.qy> It appears that Lars Brinkhoff said: >G. Branden Robinson wrote: >> well after Unix had fledged, its developers at CSRC found it necessary >> and/or desirable to borrow back a Multics concept: they named it mmap(). > >It has been argued the borrowing was from TENEX. TENEX in turn would >have been influenced by Multics and the SDS 940 timesharing system. mmap() is a lot more like Tenex than it is like Multics. Tenex ran on a PDP-10 with a flat 18 bit address space, and you could map files to places in that address space. It didn't have any of the Multics segment stuff where a process was a collection of segments and you made a segment addressable which implicitly mapped the whole segment. From g.branden.robinson at gmail.com Mon Dec 16 06:17:21 2024 From: g.branden.robinson at gmail.com (G. Branden Robinson) Date: Sun, 15 Dec 2024 14:17:21 -0600 Subject: [TUHS] M macros, wasRe: SCCS In-Reply-To: <20241215200920.9D746AE68463@ary.qy> References: <20241215200920.9D746AE68463@ary.qy> Message-ID: <20241215201721.2kx77jofeyvztdj2@illithid> At 2024-12-15T15:09:20-0500, John Levine wrote: > >On Sun, Dec 15, 2024 at 10:49 AM Marc Rochkind wrote: > >> Incidentally, UNIX had a different language-independent macro processor > >> called m6. ... > > >Correction: The version of M6 for UNIX was called M4. Maybe because it was > >only 2/3 as complete? > > The Wikipedia article on macroprocessors says that M6 was written in > the 1960s by McIlroy, Morris, and Hall, based on GPM and Trac, written > in Fortran and ported to v2 Unix. > > M4 was written in the 1970s by Kernighan and Ritchie in C and is still > around, notably as impenetrable magic in GNU autoconfig and sendmail > config files. It looks a lot like GPM. Being aware of its reputation, I had some trepidation about using it, and found its impenetrability to be overstated. For a few years now I've used it to generate two man pages from a single source: groff_man(7) and groff_man_style(7). https://git.savannah.gnu.org/cgit/groff.git/tree/tmac/groff_man.7.man.in?h=1.23.0 The only thing I stubbed my toe on is m4's appropriation of common English words for its command language. A prefix sigil before such words would have been a better choice. But I got around that, too. https://git.savannah.gnu.org/cgit/groff.git/tree/tmac/tmac.am?h=1.23.0#n252 Regards, Branden -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From crossd at gmail.com Mon Dec 16 06:24:58 2024 From: crossd at gmail.com (Dan Cross) Date: Sun, 15 Dec 2024 15:24:58 -0500 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <08a76bf2-5785-d293-7d84-ef17f2d11263@makerlisp.com> References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> <7wzfkxjtqi.fsf@junk.nocrew.org> <20241215094358.ilgv62mui7tyxhdo@illithid> <5e69ed7b-b2be-442a-9b49-6884457084f7@case.edu> <08a76bf2-5785-d293-7d84-ef17f2d11263@makerlisp.com> Message-ID: On Sun, Dec 15, 2024 at 2:12 PM Luther Johnson wrote: > I don't know where mmap showed up in vanilla BSD distributions, but it > was in SunOS 4.0, I used it. kern_mman.c shows up in 4.1c, but is behind an #ifdef; I gather it wasn't fully implemented. It was described in 4.2, as mentioned, but wasn't "fully functional" until Net/2 (according to mmap(2), anyway). Sun's version was an independent implementation. As mentioned by Chet, the interface is modeled after the PMAP call on TENEX/TOPS-20. The virtual memory system on TENEX, in turn, was at least partially influenced by Multics, so there is some Multics intellectual DNA (perhaps mutated :-)) in mmap(), though it's indirect. - Dan C. > To the other points, the Multics > "one-level-store" idea was very powerful, and was an important influence > in Apollo Domain/OS, and it was probably there in Prime Computer's OS too. > > On 12/15/2024 12:02 PM, Chet Ramey via TUHS wrote: > > On 12/15/24 4:43 AM, G. Branden Robinson wrote: > >> At 2024-12-15T07:22:29+0000, Lars Brinkhoff wrote: > > > >> Maybe I would, in fact, hate using Multics. But I can't forget that > >> well after Unix had fledged, its developers at CSRC found it necessary > >> and/or desirable to borrow back a Multics concept: they named it mmap(). > > > > I think it was primarily inspired by TENEX/TOPS-20, and it was the CSRG > > who proposed it. The concepts are described in CSRG TR/4 and mmap() > > ended up described in the 4.2BSD manual, but not implemented. > > > > I don't think Reiser's VM system at the Labs included mmap(), but I think > > it did have something similar. Of course, that never really saw the light > > of day. > > > From pugs78 at gmail.com Mon Dec 16 06:30:19 2024 From: pugs78 at gmail.com (Tom Lyon) Date: Sun, 15 Dec 2024 12:30:19 -0800 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> <7wzfkxjtqi.fsf@junk.nocrew.org> <20241215094358.ilgv62mui7tyxhdo@illithid> <5e69ed7b-b2be-442a-9b49-6884457084f7@case.edu> <08a76bf2-5785-d293-7d84-ef17f2d11263@makerlisp.com> Message-ID: Yeah, the big changes in SunOS 4.0 were all around mmap and shared libraries. But I did a tiny subset of mmap - for mapping device registers only - in SunOS 2 (or 3?). But I used the CSRG spec for mmap. On Sun, Dec 15, 2024 at 12:25 PM Dan Cross wrote: > On Sun, Dec 15, 2024 at 2:12 PM Luther Johnson > wrote: > > I don't know where mmap showed up in vanilla BSD distributions, but it > > was in SunOS 4.0, I used it. > > kern_mman.c shows up in 4.1c, but is behind an #ifdef; I gather it > wasn't fully implemented. It was described in 4.2, as mentioned, but > wasn't "fully functional" until Net/2 (according to mmap(2), anyway). > > Sun's version was an independent implementation. > > As mentioned by Chet, the interface is modeled after the PMAP call on > TENEX/TOPS-20. The virtual memory system on TENEX, in turn, was at > least partially influenced by Multics, so there is some Multics > intellectual DNA (perhaps mutated :-)) in mmap(), though it's > indirect. > > - Dan C. > > > To the other points, the Multics > > "one-level-store" idea was very powerful, and was an important influence > > in Apollo Domain/OS, and it was probably there in Prime Computer's OS > too. > > > > On 12/15/2024 12:02 PM, Chet Ramey via TUHS wrote: > > > On 12/15/24 4:43 AM, G. Branden Robinson wrote: > > >> At 2024-12-15T07:22:29+0000, Lars Brinkhoff wrote: > > > > > >> Maybe I would, in fact, hate using Multics. But I can't forget that > > >> well after Unix had fledged, its developers at CSRC found it necessary > > >> and/or desirable to borrow back a Multics concept: they named it > mmap(). > > > > > > I think it was primarily inspired by TENEX/TOPS-20, and it was the CSRG > > > who proposed it. The concepts are described in CSRG TR/4 and mmap() > > > ended up described in the 4.2BSD manual, but not implemented. > > > > > > I don't think Reiser's VM system at the Labs included mmap(), but I > think > > > it did have something similar. Of course, that never really saw the > light > > > of day. > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From douglas.mcilroy at dartmouth.edu Mon Dec 16 06:48:40 2024 From: douglas.mcilroy at dartmouth.edu (Douglas McIlroy) Date: Sun, 15 Dec 2024 15:48:40 -0500 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git Message-ID: > well after Unix had fledged, its developers at CSRC found it necessary > and/or desirable to borrow back a Multics concept: they named it mmap(). As far as I know no Research version of Unix ever had mmap. Multics had a segmented universal memory. A process incorporated segments into its address space The universal memory was normally addressed via a hierachical segment-name directory. With enhancement to provide for multisegment "files", the directory could serve as a file system and file I/O became data transfer between segments. Unix originally imitated the Multics file system, but not the universal memory. mmap(2) weakly imitates universal memory by allowing a process to nominally incorporate a portion of a file into the process address space at page-level granularity. However, an update is guaranteed to be visible to the file and other processes only upon specific request. Does anyone know whether there are implementations of mmap that do transparent file sharing? It seems to me that should be possible by making the buffer cache share pages with mmapping processes. Doug From lm at mcvoy.com Mon Dec 16 06:57:51 2024 From: lm at mcvoy.com (Larry McVoy) Date: Sun, 15 Dec 2024 12:57:51 -0800 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: <20241215205751.GA17570@mcvoy.com> On Sun, Dec 15, 2024 at 03:48:40PM -0500, Douglas McIlroy wrote: > Unix originally imitated the Multics file system, but not the universal > memory. mmap(2) weakly imitates universal memory by allowing a process > to nominally incorporate a portion of a file into the process address space > at page-level granularity. However, an update is guaranteed to be visible > to the file and other processes only upon specific request. That's not true with Sun's mmap(). It's coherent across process boundaries and it's in sync with read()/write(). This is because Sun got rid of the buffer cache and _only_ did file IO to/from the page cache. You mapped the actual page into your address space, if one process wrote it, the other process will see it. > Does anyone know whether there are implementations of mmap that > do transparent file sharing? It seems to me that should be possible by > making the buffer cache share pages with mmapping processes. SunOS 4.0 did what I believe you are asking. http://mcvoy.com/lm/papers/SunOS.vm_arch.pdf http://mcvoy.com/lm/papers/SunOS.vm_impl.pdf -- --- Larry McVoy Retired to fishing http://www.mcvoy.com/lm/boat From sauer at technologists.com Mon Dec 16 07:36:39 2024 From: sauer at technologists.com (Charles H Sauer (he/him)) Date: Sun, 15 Dec 2024 15:36:39 -0600 Subject: [TUHS] M macros, wasRe: SCCS In-Reply-To: <20241215201721.2kx77jofeyvztdj2@illithid> References: <20241215200920.9D746AE68463@ary.qy> <20241215201721.2kx77jofeyvztdj2@illithid> Message-ID: On 12/15/2024 2:17 PM, G. Branden Robinson wrote: > At 2024-12-15T15:09:20-0500, John Levine wrote: >>> On Sun, Dec 15, 2024 at 10:49 AM Marc Rochkind wrote: >>>> Incidentally, UNIX had a different language-independent macro processor >>>> called m6. ... >> >>> Correction: The version of M6 for UNIX was called M4. Maybe because it was >>> only 2/3 as complete? >> >> The Wikipedia article on macroprocessors says that M6 was written in >> the 1960s by McIlroy, Morris, and Hall, based on GPM and Trac, written >> in Fortran and ported to v2 Unix. >> >> M4 was written in the 1970s by Kernighan and Ritchie in C and is still >> around, notably as impenetrable magic in GNU autoconfig and sendmail >> config files. It looks a lot like GPM. > > Being aware of its reputation, I had some trepidation about using it, > and found its impenetrability to be overstated. > > For a few years now I've used it to generate two man pages from a single > source: groff_man(7) and groff_man_style(7). > > https://git.savannah.gnu.org/cgit/groff.git/tree/tmac/groff_man.7.man.in?h=1.23.0 > > The only thing I stubbed my toe on is m4's appropriation of common > English words for its command language. A prefix sigil before such > words would have been a better choice. But I got around that, too. > > https://git.savannah.gnu.org/cgit/groff.git/tree/tmac/tmac.am?h=1.23.0#n252 > > Regards, > Branden In 1997, when CSS was just beginning, long before https://en.wikipedia.org/wiki/M4_%28computer_language%29 was started in 2004, and subsequently illustrated m4 macros for creating HTML, I started using m4 macros extensively to define Web pages in such a matter that they could mimic appearance of other pages, and taught others at our startup to use those macros, so that our customers could use our software while retaining appearance consistent with the rest of their pages. Another way to think of the macros is that they comprise a static content management system – the content is stored in m4 files, which are transformed into HTML in advance, vs. more dynamic page generation in a typical content management system. https://web.archive.org/web/19990125090055/http://hire.com/ describes the software. https://web.archive.org/web/19980209192647/http://www.eds.com/careers/overview/cr_overview.shtml, https://web.archive.org/web/19990224005553/http://world4.hire.com/SVB/, and https://web.archive.org/web/19990422144616/http://www.careerstop.org/job.htm show remnants of customer pages created with those m4 macros. More at https://technologists.com/notes/2007/11/02/css-a-mans-got-to-know-his-limitations-2/ Charlie -- voice: +1.512.784.7526 e-mail: sauer at technologists.com fax: +1.512.346.5240 Web: https://technologists.com/sauer/ Facebook/Google/LinkedIn/Twitter: CharlesHSauer From johnl at taugh.com Mon Dec 16 09:05:07 2024 From: johnl at taugh.com (John Levine) Date: 15 Dec 2024 18:05:07 -0500 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: <20241215230508.6901BAE6D3A4@ary.qy> It appears that Douglas McIlroy said: >Unix originally imitated the Multics file system, but not the universal >memory. mmap(2) weakly imitates universal memory by allowing a process >to nominally incorporate a portion of a file into the process address space >at page-level granularity. However, an update is guaranteed to be visible >to the file and other processes only upon specific request. > >Does anyone know whether there are implementations of mmap that >do transparent file sharing? It seems to me that should be possible by >making the buffer cache share pages with mmapping processes. These days they all do. The POSIX rationale says: A memory object can be concurrently mapped into the address space of one or more processes. The mmap( ) and munmap( ) functions allow a process to manipulate their address space by mapping portions of memory objects into it and removing them from it. When multiple processes map the same memory object, they can share access to the underlying data. "Memory object" includes disk files. There are MAP_SHARED and MAP_PRIVATE flags that say whether changes are written through or copy-on-write to local private pages. R's, John From mrochkind at gmail.com Mon Dec 16 11:51:42 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Sun, 15 Dec 2024 18:51:42 -0700 Subject: [TUHS] M macros, wasRe: SCCS In-Reply-To: References: <20241215200920.9D746AE68463@ary.qy> <20241215201721.2kx77jofeyvztdj2@illithid> Message-ID: I found this TUHS thread from 2019: https://tuhs.org/mailman3/hyperkitty/list/tuhs at tuhs.org/thread/GPFOZNHNX2JOPPTPJEPRILDIT5O7N6QS/ in which Andy Hall's 1972 memo on M6 is referenced: https://plan9.io/cm/cs/cstr/2.pdf Marc On Sun, Dec 15, 2024 at 2:53 PM Charles H Sauer (he/him) < sauer at technologists.com> wrote: > On 12/15/2024 2:17 PM, G. Branden Robinson wrote: > > At 2024-12-15T15:09:20-0500, John Levine wrote: > >>> On Sun, Dec 15, 2024 at 10:49 AM Marc Rochkind > wrote: > >>>> Incidentally, UNIX had a different language-independent macro > processor > >>>> called m6. ... > >> > >>> Correction: The version of M6 for UNIX was called M4. Maybe because it > was > >>> only 2/3 as complete? > >> > >> The Wikipedia article on macroprocessors says that M6 was written in > >> the 1960s by McIlroy, Morris, and Hall, based on GPM and Trac, written > >> in Fortran and ported to v2 Unix. > >> > >> M4 was written in the 1970s by Kernighan and Ritchie in C and is still > >> around, notably as impenetrable magic in GNU autoconfig and sendmail > >> config files. It looks a lot like GPM. > > > > Being aware of its reputation, I had some trepidation about using it, > > and found its impenetrability to be overstated. > > > > For a few years now I've used it to generate two man pages from a single > > source: groff_man(7) and groff_man_style(7). > > > > > https://git.savannah.gnu.org/cgit/groff.git/tree/tmac/groff_man.7.man.in?h=1.23.0 > > > > The only thing I stubbed my toe on is m4's appropriation of common > > English words for its command language. A prefix sigil before such > > words would have been a better choice. But I got around that, too. > > > > > https://git.savannah.gnu.org/cgit/groff.git/tree/tmac/tmac.am?h=1.23.0#n252 > > > > Regards, > > Branden > > In 1997, when CSS was just beginning, long before > https://en.wikipedia.org/wiki/M4_%28computer_language%29 was started in > 2004, and subsequently illustrated m4 macros for creating HTML, I > started using m4 macros extensively to define Web pages in such a matter > that they could mimic appearance of other pages, and taught others at > our startup to use those macros, so that our customers could use our > software while retaining appearance consistent with the rest of their > pages. Another way to think of the macros is that they comprise a static > content management system – the content is stored in m4 files, which are > transformed into HTML in advance, vs. more dynamic page generation in a > typical content management system. > https://web.archive.org/web/19990125090055/http://hire.com/ describes > the software. > > https://web.archive.org/web/19980209192647/http://www.eds.com/careers/overview/cr_overview.shtml, > > https://web.archive.org/web/19990224005553/http://world4.hire.com/SVB/, > and > > https://web.archive.org/web/19990422144616/http://www.careerstop.org/job.htm > show remnants of customer pages created with those m4 macros. > > More at > > https://technologists.com/notes/2007/11/02/css-a-mans-got-to-know-his-limitations-2/ > > > Charlie > > > -- > voice: +1.512.784.7526 e-mail: sauer at technologists.com > fax: +1.512.346.5240 Web: https://technologists.com/sauer/ > Facebook/Google/LinkedIn/Twitter > : > CharlesHSauer > > -- *My new email address is mrochkind at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnold at skeeve.com Mon Dec 16 16:39:05 2024 From: arnold at skeeve.com (arnold at skeeve.com) Date: Sun, 15 Dec 2024 23:39:05 -0700 Subject: [TUHS] M macros, wasRe: SCCS In-Reply-To: <20241215200920.9D746AE68463@ary.qy> References: <20241215200920.9D746AE68463@ary.qy> Message-ID: <202412160639.4BG6d55C068414@freefriends.org> "John Levine" wrote: > M4 was written in the 1970s by Kernighan and Ritchie in C ... In private mail, BWK told me that it was DMR who wrote m4. He then reimplemented it in Ratfor for "Software Tools". Just to set the record straight. :-) Arnold From douglas.mcilroy at dartmouth.edu Tue Dec 17 00:21:57 2024 From: douglas.mcilroy at dartmouth.edu (Douglas McIlroy) Date: Mon, 16 Dec 2024 09:21:57 -0500 Subject: [TUHS] M macros, wasRe: SCCS Message-ID: > "John Levine" wrote: >> M4 was written in the 1970s by Kernighan and Ritchie in C ... > In private mail, BWK told me that it was DMR who wrote m4. He t> hen reimplemented it in Ratfor for "Software Tools". > Arnold The book says colorfully, "... [and] we are grateful to him for letting us steal it." Doug From douglas.mcilroy at dartmouth.edu Tue Dec 17 01:40:34 2024 From: douglas.mcilroy at dartmouth.edu (Douglas McIlroy) Date: Mon, 16 Dec 2024 10:40:34 -0500 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git Message-ID: >> Does anyone know whether there are implementations of mmap that >> do transparent file sharing? It seems to me that should be possible by >> making the buffer cache share pages with mmapping processes. > These days they all do. The POSIX rationale says: > ... When multiple processes map the same memory object, they can > share access to the underlying data. Notice the weasel word "can". It is not guaranteed that they will do so automatically without delay. Apparently each process may have a physically distinct copy of the data, not shared access to a single location. The Linux man page mmap(2), for example, makes it very clear that mmap has a cache-coherence problem, at least in that system. The existence of msync(2) is a visible symptom of the problem. [Weasel words of my own: I have not read the POSIX definition of mmap.] Doug From clemc at ccc.com Tue Dec 17 02:09:42 2024 From: clemc at ccc.com (Clem Cole) Date: Mon, 16 Dec 2024 11:09:42 -0500 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: On Mon, Dec 16, 2024 at 10:40 AM Douglas McIlroy < douglas.mcilroy at dartmouth.edu> wrote: > >> Does anyone know whether there are implementations of mmap that > >> do transparent file sharing? It seems to me that should be possible by > >> making the buffer cache share pages with mmapping processes. > > > These days they all do. The POSIX rationale says: > > > ... When multiple processes map the same memory object, they can > > share access to the underlying data. > > Notice the weasel word "can". It is not guaranteed that they will do so > automatically without delay. Apparently each process may have a physically > distinct copy of the data, not shared access to a single location. > Hmmm — how did that make it through the IEEE editing process? However, that is not part of the standard if it's in the Rationale - just an explanation of the standard - which I want want look at more carefully to see what it says about what is required of mmap(2). That said, as Heinz and the rest of us who worked on the original version of what would come to POSIX can tell you, the word "can" was always a no-no. The word "SHALL" is the the official IEEE word, and if you use "SHOULD," even in the rationale and it was very much frowned upon. I'll have try dig up the troff sources to an early draft of the original and do a "grep -i should * | wc -l" through it. I bet the number is very small and "can" is zero. That said, it has been a few years since I have read a current draft, but less the rationale section (which is not the standard itself). Clem Clem ᐧ -------------- next part -------------- An HTML attachment was scrubbed... URL: From johnl at taugh.com Tue Dec 17 02:56:06 2024 From: johnl at taugh.com (John R Levine) Date: 16 Dec 2024 11:56:06 -0500 Subject: [TUHS] M macros, wasRe: SCCS In-Reply-To: <202412160639.4BG6d55C068414@freefriends.org> References: <20241215200920.9D746AE68463@ary.qy> <202412160639.4BG6d55C068414@freefriends.org> Message-ID: <58ea803a-545d-784b-f4fe-9df54748e8d0@taugh.com> On Sun, 15 Dec 2024, arnold at skeeve.com wrote: > "John Levine" wrote: > >> M4 was written in the 1970s by Kernighan and Ritchie in C ... > > In private mail, BWK told me that it was DMR who wrote m4. He > then reimplemented it in Ratfor for "Software Tools". > > Just to set the record straight. :-) You might want to update the Wikipedia articles. Regards, John Levine, johnl at taugh.com, Taughannock Networks, Trumansburg NY Please consider the environment before reading this e-mail. https://jl.ly From steffen at sdaoden.eu Tue Dec 17 03:55:29 2024 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Mon, 16 Dec 2024 18:55:29 +0100 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: <20241216175529.Qud9TgZk@steffen%sdaoden.eu> Marc Rochkind wrote in : |On Sat, Dec 14, 2024 at 6:43 PM Greg A. Woods wrote: |> This is more of an aside, but as a long time and ongoing SCCS user I've |> wondered about the vc(1) command included in the SCCS tool suite. |> |> So I was hoping you (or anyone else reading) might be able to shed some |> light on its origins and maybe give, or point to, some examples of how |> it was intended to be used, or indeed how it ended up being used. | |I had completely forgotten about the vc command until I read this post. Vc |was a language-independent macro processor mostly concerned with excluding |or including lines of text based on the evaluation of logical expressions. | |Even though "vc" stands for "version control," it has nothing to do with |version control as that phrase is used nowadays. In the early 1970s where I |worked at Bell Labs, we (or at least I) used the term "release" to refer to |the different forms of a file as development proceeded, and "version" to |refer to ongoing variants of the file. For example, there might be a |version of some program for Southwestern Bell and a different version for |New York Telephone. Those two versions were contained in the same identical |sets of source files. | |I just looked again at my 1975 paper and I don't think the word "version" |appears anywhere. The terms "release" and "level" appear. | |I designed vc, but it was implemented by a college student who was with us |for the Summer. I credit a woman named Sabrina Feczko in my paper, and I |think it was her. As I mentioned, vc was totally standalone so Sabrina |didn't have to concern herself with all the squirrely code in the main part |of SCCS. | |Vc disappeared at some point. I don't think it was ever used, as it was Jörg's (Schillig) SCCS ships with vc. (His version lost some limiting restrictions according to announcements, for example line length.) I have never used vc myself, so i cannot give examples. (He worked quite intensively on his SCCS and the v6 file format in his last year, wrote manuals, created tests etc. I do not think his changeset approach was completed.) |intended for mainframe programs that were being developed (but not |compiled) on PWB/UNIX. For UNIX development, C had its own macro processor. ... --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) | |And in Fall, feel "The Dropbear Bard"s ball(s). | |The banded bear |without a care, |Banged on himself for e'er and e'er | |Farewell, dear collar bear From tytso at mit.edu Tue Dec 17 05:02:56 2024 From: tytso at mit.edu (Theodore Ts'o) Date: Mon, 16 Dec 2024 14:02:56 -0500 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: <20241216190256.GA78919@mit.edu> On Mon, Dec 16, 2024 at 10:40:34AM -0500, Douglas McIlroy wrote: > >> Does anyone know whether there are implementations of mmap that > >> do transparent file sharing? It seems to me that should be possible by > >> making the buffer cache share pages with mmapping processes. > > > These days they all do. The POSIX rationale says: > > > ... When multiple processes map the same memory object, they can > > share access to the underlying data. I don't see this language in the 2024 version of POSIX (IEEE Std 1003.1-2024). > Notice the weasel word "can". It is not guaranteed that they will do so > automatically without delay. Apparently each process may have a physically > distinct copy of the data, not shared access to a single location. > > The Linux man page mmap(2), for example, makes it very clear that mmap > has a cache-coherence problem, at least in that system. The existence > of msync(2) is a visible symptom of the problem. Huh? What language are you talking about? msync(2) is about when dirty data is written to stable storage (e.g., written back out to disk). It works much like fsync(2). There is a cache-coherency problem between what's in the buffer/page cache (Linux has a unified page cache, so file-backed blocks are not cached in the buffer cache; only the page cache) and O_DIRECT. Some OS's will tell you that there is no coherency guarantees at all. Linux will give a best-efforts attempt at coherency which is to say that before doing an O_DIRECT read, any dirty pages will be written out to disk before the O_DIRECT read is allowed to proceed, and before doing an O_DIRECT write, any underlying pages in the page cache will be invalidated. It is not 100% guaranteed to be coherent, and system which try to mix simultaneous buffered and O_DIRECT I/O may not necessarily get what they want. However, if what you are talking about is multiple processes mmap'ing the same file, they *will* get the same page in the page cache mapped into the page tables, which means that writes by one process will be immediately seen by another process (modulo CPU cache coherency guarantees, of course; Intel is pretty good; with a weekly consistent architecture like Alpha, your mileage may vary unless you know what you are doing and write your assembly code accordingly). Cheers, - Ted From johnl at taugh.com Tue Dec 17 05:08:43 2024 From: johnl at taugh.com (John Levine) Date: 16 Dec 2024 14:08:43 -0500 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: <20241216190843.D9C08AE8CC9D@ary.qy> It appears that Douglas McIlroy said: >[Weasel words of my own: I have not read the POSIX definition of mmap.] We're in luck, I have it right here. On pages 529-530: 2.8.3.2 Memory Mapped Files Range memory mapping operations are defined in terms of pages. Implementations may restrict the size and alignment of range mappings to be on page-size boundaries. The page size, in bytes, is the value of the configurable system variable {PAGESIZE}. If an implementation has no restrictions on size or alignment, it may specify a 1-byte page size. Memory mapped files provide a mechanism that allows a process to access files by directly incorporating file data into its address space. Once a file is mapped into a process address space, the data can be manipulated as memory. If more than one process maps a file, its contents are shared among them. If the mappings allow shared write access, then data written into the memory object through the address space of one process appears in the address spaces of all processes that similarly map the same portion of the memory object. The mmap() man page on debian says this, which seems clear enough: MAP_SHARED Share this mapping. Updates to the mapping are visible to other processes mapping the same region, and (in the case of file-backed mappings) are carried through to the underlying file. (To precisely control when updates are carried through to the underlying file requires the use of msync(2).) The msync() call is a minor fudge that lets you control how briskly changes are written back to disk. But it seems clear enough that all the processes that have a page mapped see the same page, and msync is for databases that want to bs sure that changes are committed to permananent storage (a term in the POSIX spec) before unlocking or unmapping. R's, John PS: I can believe there are some versions of linux that screwed up disk cache coherency, but that just means they don't properly implement the spec, not for the first time. I mean, it's not *that* hard to make all the maps point to the same physical page frame, even on a machine like POWER with reverse page maps. From steffen at sdaoden.eu Tue Dec 17 05:40:43 2024 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Mon, 16 Dec 2024 20:40:43 +0100 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: <20241216194043.PqDb-or7@steffen%sdaoden.eu> Clem Cole wrote in : |On Mon, Dec 16, 2024 at 10:40 AM Douglas McIlroy < |douglas.mcilroy at dartmouth.edu> wrote: | |>>> Does anyone know whether there are implementations of mmap that |>>> do transparent file sharing? It seems to me that should be possible by |>>> making the buffer cache share pages with mmapping processes. |> |>> These days they all do. The POSIX rationale says: |> |>> ... When multiple processes map the same memory object, they can |>> share access to the underlying data. |> |> Notice the weasel word "can". It is not guaranteed that they will do so |> automatically without delay. Apparently each process may have a physical\ |> ly |> distinct copy of the data, not shared access to a single location. |> |Hmmm — how did that make it through the IEEE editing process? However, that |is not part of the standard if it's in the Rationale - just an explanation |of the standard - which I want want look at more carefully to see what it |says about what is required of mmap(2). | |That said, as Heinz and the rest of us who worked on the original version |of what would come to POSIX can tell you, the word "can" was always a |no-no. The word "SHALL" is the the official IEEE word, and if you use |"SHOULD," even in the rationale and it was very much frowned upon. I'll They are all lowercase nowadays. |have try dig up the troff sources to an early draft of the original and do |a "grep -i should * | wc -l" through it. I bet the number is very small |and "can" is zero. They offer lots of hints like "can be implemented efficiently" or "conforming implementations cannot count on", which turns "can" into a "dying butterfly". (In the rationale's.) |That said, it has been a few years since I have read a current draft, but |less the rationale section (which is not the standard itself). The 2024 version is 4057 pages, all in all. It would -- in my opinion -- make a nice discussion whether it would be better for anybody, including engineers, to take the time to read that amount of cultural foundation, real literature that is, instead of a technical standard. (I admit i personally perform standard hopping and have at times some spare time for other things, because if i fail it currently endangers noone for one, and when i fall i stand up anyway, unless i do no more, but then having lived in mindsets++ of fantastic literats.) |Clem --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) | |And in Fall, feel "The Dropbear Bard"s ball(s). | |The banded bear |without a care, |Banged on himself for e'er and e'er | |Farewell, dear collar bear From tuhs at tuhs.org Tue Dec 17 05:45:08 2024 From: tuhs at tuhs.org (Chet Ramey via TUHS) Date: Mon, 16 Dec 2024 14:45:08 -0500 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241216190843.D9C08AE8CC9D@ary.qy> References: <20241216190843.D9C08AE8CC9D@ary.qy> Message-ID: <23e265fa-cf0d-4fe4-9b23-f54aa3e0ee9e@case.edu> On 12/16/24 2:08 PM, John Levine wrote: > It appears that Douglas McIlroy said: >> [Weasel words of my own: I have not read the POSIX definition of mmap.] > > We're in luck, I have it right here. On pages 529-530: Here are the current (POSIX.1-2024) versions of the appropriate definitions and the mmap() description, for those who want to read along. https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap03.html#tag_03_200 https://pubs.opengroup.org/onlinepubs/9799919799/functions/mmap.html#tag_17_345 > > 2.8.3.2 Memory Mapped Files Is that from the Rationale? Here's the current version. https://pubs.opengroup.org/onlinepubs/9799919799/xrat/V4_xsh_chap01.html#tag_22_02_08_13 -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature.asc Type: application/pgp-signature Size: 203 bytes Desc: OpenPGP digital signature URL: From steffen at sdaoden.eu Tue Dec 17 05:47:10 2024 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Mon, 16 Dec 2024 20:47:10 +0100 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241216194043.PqDb-or7@steffen%sdaoden.eu> References: <20241216194043.PqDb-or7@steffen%sdaoden.eu> Message-ID: <20241216194710.2VuYsfJn@steffen%sdaoden.eu> Steffen Nurpmeso wrote in <20241216194043.PqDb-or7 at steffen%sdaoden.eu>: |Clem Cole wrote in | : ||On Mon, Dec 16, 2024 at 10:40 AM Douglas McIlroy < ||douglas.mcilroy at dartmouth.edu> wrote: || ||>>> Does anyone know whether there are implementations of mmap that ||>>> do transparent file sharing? It seems to me that should be possible by ||>>> making the buffer cache share pages with mmapping processes. ||> ||>> These days they all do. The POSIX rationale says: ||> ||>> ... When multiple processes map the same memory object, they can ||>> share access to the underlying data. ||> ||> Notice the weasel word "can". It is not guaranteed that they will do so ||> automatically without delay. Apparently each process may have a \ ||> physical\ ||> ly ||> distinct copy of the data, not shared access to a single location. I think it was in FreeBSD when sed(1) once got an optimization to mmap(1) data, but it was reverted because of crashes caused by concurrent modifications. (There definetely was something like this, but FreeBSD / sed is nothing but what i remember.) But there is no weasel word in the real standard text, but lots of "shall". The only "can" there are There may be implementation-defined limits on the number of memory regions that can be mapped (per process or per system). and If such a limit is imposed, whether the number of memory regions that can be mapped by a process is decreased by the use of shmat( ) is implementation-defined. which are no "weasel-can's". --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) | |And in Fall, feel "The Dropbear Bard"s ball(s). | |The banded bear |without a care, |Banged on himself for e'er and e'er | |Farewell, dear collar bear From clemc at ccc.com Tue Dec 17 06:58:53 2024 From: clemc at ccc.com (Clem Cole) Date: Mon, 16 Dec 2024 15:58:53 -0500 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241216194043.PqDb-or7@steffen%sdaoden.eu> References: <20241216194043.PqDb-or7@steffen%sdaoden.eu> Message-ID: On Mon, Dec 16, 2024 at 2:40 PM Steffen Nurpmeso wrote: > > > They are all lowercase nowadays. > They were always case-sensitive to use. I used upper case to make it stand out in my message. > > |have try dig up the troff sources to an early draft of the original and > do > |a "grep -i should * | wc -l" through it. I bet the number is very small > |and "can" is zero. > > They offer lots of hints like "can be implemented efficiently" or > "conforming implementations cannot count on", which turns "can" > into a "dying butterfly". (In the rationale's.) > It always say. *"conforming implementations shall not count on."* The standard has to be as precise as it can be. If there are implementation grey areas, then the standard needs to state that too. Again - we were careful when we wrote original documents (and had lots of arguments as you can imagine). As for what you read, I'm not worried. The fact the over time, standards stop being what they were originally intended and take a life of their own. POSIX is less of an issue than say, C++ or even FORTRAN for that matter. But over time, a new group of people want to "make their mark." Some of us grew tired of arguing. We got what we originally set out to do, and I left the POSIX work soon after the *.2 began. I worked through one draft of it, but was not there for approval, although I helped with *.4 at one point. Someone implementing something new, be it a language or a system, should be familiar with the standards. They is a lot be learned both good and bad. Reinventing things, particularly bad ideas, is hardly for the greater good (although the implementor might find it fun). e.g. C++, IMO, is a great example of what >>not<< to do. The core POSIX.1 interface, think is excellent and gets to the point. After that, YMMV and the value you get from it also is a bit variable. But thinking the world should be stagnant and believing that C or UNIX (or modern Unix, a.k.a. Linux) is "the end" is hardly a good idea either. So, reading and learning about them does have some merit. The question is how much and how far to go. Clem ᐧ -------------- next part -------------- An HTML attachment was scrubbed... URL: From johnl at taugh.com Tue Dec 17 07:08:23 2024 From: johnl at taugh.com (John R Levine) Date: 16 Dec 2024 16:08:23 -0500 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: <20241216190843.D9C08AE8CC9D@ary.qy> Message-ID: <2af0cc7c-d0db-312f-df89-036713b8847d@taugh.com> On Mon, 16 Dec 2024, Konstantin Belousov wrote: > On Mon, Dec 16, 2024 at 02:08:43PM -0500, John Levine wrote: >> PS: I can believe there are some versions of linux that screwed up disk cache >> coherency, but that just means they don't properly implement the spec, not for >> the first time. I mean, it's not *that* hard to make all the maps point to the >> same physical page frame, even on a machine like POWER with reverse page maps. > > This is not enough. There are (were ?) architectures, typically with the > virtually addressed caches, which require all mappings of the same page > to be suitably aligned, at least. ... > > If addresses of different mappings are not aligned, caches were not coherent. I think we're in "so don't do that" territory. mmap() normally lets the system pick the memory address to map so it can pick something suitably aligned. You can pass the MAP_FIXED flag to tell it to map at a particular address, but it can return EINVAL if the address doesn't work. The POSIX description says "The use of MAP_FIXED is discouraged, as it may prevent an implementation from making the most effective use of resources." It's not always trivial to make this work. On systems with reverse maps, a physical page can only be mapped to one virtual address at a time, so for shared pages it has to mark all of the aliases nonresident and on a fault remap the page into the map of the process that is running. But it's not rocket science, either. R's, John From steffen at sdaoden.eu Tue Dec 17 07:50:40 2024 From: steffen at sdaoden.eu (Steffen Nurpmeso) Date: Mon, 16 Dec 2024 22:50:40 +0100 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: <20241216194043.PqDb-or7@steffen%sdaoden.eu> Message-ID: <20241216215040.6FWLhbCs@steffen%sdaoden.eu> Clem Cole wrote in : |On Mon, Dec 16, 2024 at 2:40 PM Steffen Nurpmeso \ |wrote: |> They are all lowercase nowadays. |> |They were always case-sensitive to use. I used upper case to make it stand |out in my message. Ah, ok. In the last years i am reading lots of IETF outcome, and for them this makes a world of a difference. |>|have try dig up the troff sources to an early draft of the original and |> do |>|a "grep -i should * | wc -l" through it. I bet the number is very small |>|and "can" is zero. |> |> They offer lots of hints like "can be implemented efficiently" or |> "conforming implementations cannot count on", which turns "can" |> into a "dying butterfly". (In the rationale's.) |> |It always say. *"conforming implementations shall not count on."* |The standard has to be as precise as it can be. If there are |implementation grey areas, then the standard needs to state that too. | |Again - we were careful when we wrote original documents (and had lots of |arguments as you can imagine). | | As for what you read, I'm not worried. The fact the over time, standards |stop being what they were originally intended and take a life of their |own. POSIX is less of an issue than say, C++ or even FORTRAN for that |matter. But over time, a new group of people want to "make their mark." ..i would even put on top that if someone does *not* want to do that, they react irritated. "Too many marks", at least in the IETF world. |Some of us grew tired of arguing. We got what we originally set out to do, |and I left the POSIX work soon after the *.2 began. I worked through one |draft of it, but was not there for approval, although I helped with *.4 at |one point. | |Someone implementing something new, be it a language or a system, should be |familiar with the standards. They is a lot be learned both good and bad. |Reinventing things, particularly bad ideas, is hardly for the greater good |(although the implementor might find it fun). e.g. C++, IMO, is a great Wise words of an, please excuse that, older man. It is surely nothing but true, but it will not quell the overflowing hormons of the young warriors, nor false prowd, nor .. etc etc etc. I do not know. One must not prevent young people from making their own experiences for sure, at best they can be guided a bit to not run into fatality dead-ends, maybe. |example of what >>not<< to do. The core POSIX.1 interface, think is |excellent and gets to the point. After that, YMMV and the value you get |from it also is a bit variable. But thinking the world should be stagnant |and believing that C or UNIX (or modern Unix, a.k.a. Linux) is "the end" is |hardly a good idea either. The holy men of India even do nothing at all! Except walking to the fountain of the holy river. And already in sight, the water drag dies into the ocean. You know, i have no idea regarding technology, it surely will iterate further. Surely the computer of for example space odysee was created after long talks with wise men, in the spirit of Lidlicker etc. The big picture. But regarding "the end", i for myself do not like the Rust programning language, regardless what you say. |So, reading and learning about them does have some merit. The question is |how much and how far to go. It is surely more than only merit. To the opposite, one should not drown in deep respect aka be slained due to the high intellectual penetration of a document like the POSIX standard, which is in the works for fourty years and has seen hundreds of highly educated people. (And at much later times, the one or the other idiot.) I, by the way, would expect the traditional answer being "learn the abc/koran/calligraphy/.. studiously while you are young, then gain the mastership and pleasure through living experience when you are older". As well as, and that is very important in my humble opinion, "if it doesn't come naturally, leave it". Ie, books are one thing, living it is another. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) | |And in Fall, feel "The Dropbear Bard"s ball(s). | |The banded bear |without a care, |Banged on himself for e'er and e'er | |Farewell, dear collar bear From tuhs at tuhs.org Tue Dec 17 08:19:15 2024 From: tuhs at tuhs.org (Bakul Shah via TUHS) Date: Mon, 16 Dec 2024 14:19:15 -0800 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: <981937D2-5AAB-4462-BBFC-262207EFC2FC@iitbombay.org> On Dec 16, 2024, at 7:40 AM, Douglas McIlroy wrote: > > >> ... When multiple processes map the same memory object, they can >> share access to the underlying data. > > Notice the weasel word "can". It is not guaranteed that they will do so > automatically without delay. Apparently each process may have a physically > distinct copy of the data, not shared access to a single location. May be this has to do with using POSIX on multiprocessor systems that are *not* cache coherent? May be less of an issue for now[1] though but weren't there such systems in the past? In any case shared access by itself is not enough for proper operation. Bakul [1] Given very high clock rates and increasing # of cores, I wonder how long before they start using CSMA/CD for coherency protocols! From johnl at taugh.com Tue Dec 17 08:43:31 2024 From: johnl at taugh.com (John Levine) Date: 16 Dec 2024 17:43:31 -0500 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <23e265fa-cf0d-4fe4-9b23-f54aa3e0ee9e@case.edu> References: <20241216190843.D9C08AE8CC9D@ary.qy> <23e265fa-cf0d-4fe4-9b23-f54aa3e0ee9e@case.edu> Message-ID: <20241216224332.0D7B6AE91FB1@ary.qy> It appears that Chet Ramey via TUHS said: >-=-=-=-=-=- >-=-=-=-=-=- >On 12/16/24 2:08 PM, John Levine wrote: >> It appears that Douglas McIlroy said: >>> [Weasel words of my own: I have not read the POSIX definition of mmap.] >> >> We're in luck, I have it right here. On pages 529-530: > >Here are the current (POSIX.1-2024) versions of the appropriate definitions >and the mmap() description, for those who want to read along. Well, that's confusing. That web site seems to have the same text as the IEEE's PDF but organized differently. >> 2.8.3.2 Memory Mapped Files > >Is that from the Rationale? No, like I said it's on pages 529-530 of the PDF version of IEEE 1003.1-2024 which I downloaded from the IEEE. Your web site has a copy here https://pubs.opengroup.org/onlinepubs/9799919799/functions/V2_chap02.html#tag_16_08 R's, John From tuhs at tuhs.org Tue Dec 17 09:19:21 2024 From: tuhs at tuhs.org (Chet Ramey via TUHS) Date: Mon, 16 Dec 2024 18:19:21 -0500 Subject: [TUHS] mmap, was SCCS, TeamWare, BitKeeper, and Git In-Reply-To: <20241216224332.0D7B6AE91FB1@ary.qy> References: <20241216190843.D9C08AE8CC9D@ary.qy> <23e265fa-cf0d-4fe4-9b23-f54aa3e0ee9e@case.edu> <20241216224332.0D7B6AE91FB1@ary.qy> Message-ID: <0358362b-f478-4705-94ba-e4d6680ab40a@case.edu> On 12/16/24 5:43 PM, John Levine wrote: >> Here are the current (POSIX.1-2024) versions of the appropriate definitions >> and the mmap() description, for those who want to read along. > > Well, that's confusing. That web site seems to have the same text as the > IEEE's PDF but organized differently. Sure, it's the Open Group. The advantage is it's public and on the web. (Plus I always have a browser tab open on it.) -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRU chet at case.edu http://tiswww.cwru.edu/~chet/ -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature.asc Type: application/pgp-signature Size: 203 bytes Desc: OpenPGP digital signature URL: From andrew at humeweb.com Tue Dec 17 10:21:47 2024 From: andrew at humeweb.com (andrew at humeweb.com) Date: Mon, 16 Dec 2024 16:21:47 -0800 Subject: [TUHS] SCCS roach motel In-Reply-To: <9838D982B117AE37BCEE88BCF8421F2D.for-standards-violators@oclsc.org> References: <9838D982B117AE37BCEE88BCF8421F2D.for-standards-violators@oclsc.org> Message-ID: indeed, it was. > On Dec 13, 2024, at 2:33 PM, Norman Wilson wrote: > > Rob Pike: > > According to the Unix room fortunes file, the actual quote is > > SCCS: the source-code motel -- your code checks in but it never checks out. Ken Thompson > > ==== > > As a Unix-room-culture aside: I believe this quote was what > inspired Andrew Hume to call his backup system the File Motel. > > Norman Wilson > Toronto ON From mrochkind at gmail.com Wed Dec 18 01:53:51 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Tue, 17 Dec 2024 08:53:51 -0700 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> <7wzfkxjtqi.fsf@junk.nocrew.org> <20241215094358.ilgv62mui7tyxhdo@illithid> Message-ID: On Mon, Dec 16, 2024 at 9:13 PM Rob Pike wrote: > Google ran on Perforce in the early days, then on an internal > reimplementation that could scale better (long story). I believe that's > still the case, as git continues to be desired by the users but unworkable > as the core repo at the required scale. Clever hackery allows git to feel > like it's being used, but it's all above what was at least at one time > Perforce. > > My info could well be out of date, though. > > -rob > > Rob, can you tell us anything about Piper? Marc -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuhs at tuhs.org Wed Dec 18 01:58:09 2024 From: tuhs at tuhs.org (=?utf-8?b?UGV0ZXIgV2VpbmJlcmdlciAo5rip5Y2a5qC8KSB2aWEgVFVIUw==?=) Date: Tue, 17 Dec 2024 10:58:09 -0500 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> <7wzfkxjtqi.fsf@junk.nocrew.org> <20241215094358.ilgv62mui7tyxhdo@illithid> Message-ID: https://cacm.acm.org/research/why-google-stores-billions-of-lines-of-code-in-a-single-repository/ to start with On Tue, Dec 17, 2024 at 10:54 AM Marc Rochkind wrote: > > > > On Mon, Dec 16, 2024 at 9:13 PM Rob Pike wrote: >> >> Google ran on Perforce in the early days, then on an internal reimplementation that could scale better (long story). I believe that's still the case, as git continues to be desired by the users but unworkable as the core repo at the required scale. Clever hackery allows git to feel like it's being used, but it's all above what was at least at one time Perforce. >> >> My info could well be out of date, though. >> >> -rob >> > Rob, can you tell us anything about Piper? > > Marc From mrochkind at gmail.com Wed Dec 18 02:12:15 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Tue, 17 Dec 2024 09:12:15 -0700 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> <7wzfkxjtqi.fsf@junk.nocrew.org> <20241215094358.ilgv62mui7tyxhdo@illithid> Message-ID: Great, Peter... exactly what I was looking for. Marc On Tue, Dec 17, 2024 at 8:58 AM Peter Weinberger (温博格) wrote: > > https://cacm.acm.org/research/why-google-stores-billions-of-lines-of-code-in-a-single-repository/ > to start with > > On Tue, Dec 17, 2024 at 10:54 AM Marc Rochkind > wrote: > > > > > > > > On Mon, Dec 16, 2024 at 9:13 PM Rob Pike wrote: > >> > >> Google ran on Perforce in the early days, then on an internal > reimplementation that could scale better (long story). I believe that's > still the case, as git continues to be desired by the users but unworkable > as the core repo at the required scale. Clever hackery allows git to feel > like it's being used, but it's all above what was at least at one time > Perforce. > >> > >> My info could well be out of date, though. > >> > >> -rob > >> > > Rob, can you tell us anything about Piper? > > > > Marc > -- *My new email address is mrochkind at gmail.com * -------------- next part -------------- An HTML attachment was scrubbed... URL: From robpike at gmail.com Wed Dec 18 07:05:29 2024 From: robpike at gmail.com (Rob Pike) Date: Wed, 18 Dec 2024 08:05:29 +1100 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: <20241214182556.xdgvuc2zw4bkj4v7@illithid> <20241214185319.GQ11590@mcvoy.com> <20241215033442.GD2472262@mit.edu> <7wzfkxjtqi.fsf@junk.nocrew.org> <20241215094358.ilgv62mui7tyxhdo@illithid> Message-ID: Not really. It was just the Perforce data model (which I still think is the nicest I've used) and UI, with a completely new implementation on Google infrastructure. To me and most others, it was the same as using Perforce. It was a remarkable achievement. -rob On Wed, Dec 18, 2024 at 2:54 AM Marc Rochkind wrote: > > > On Mon, Dec 16, 2024 at 9:13 PM Rob Pike wrote: > >> Google ran on Perforce in the early days, then on an internal >> reimplementation that could scale better (long story). I believe that's >> still the case, as git continues to be desired by the users but unworkable >> as the core repo at the required scale. Clever hackery allows git to feel >> like it's being used, but it's all above what was at least at one time >> Perforce. >> >> My info could well be out of date, though. >> >> -rob >> >> Rob, can you tell us anything about Piper? > > Marc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From woods at robohack.ca Wed Dec 18 08:28:19 2024 From: woods at robohack.ca (Greg A. Woods) Date: Tue, 17 Dec 2024 14:28:19 -0800 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: Thank you very much for your reply Marc. One thing I left unstated in my question, and indeed in my 1988 question as well, was more or less: "If vc is just a macro processor, why might I want to use it instead of any other available macro processor?" Indeed that was an even more important question back then before there was an open-source version of SCCS (with vc). Back in 1988 I only had access to a public SunOS machine, a public SCO Xenix machine, and whatever Unix(-like) machines might be available at my current workplace, and of course not all of them might have had SCCS installed. As a C programmer I was already familiar with CPP of course, and having been a Unix user since my earliest university courses I was also familiar, at least in passing in 1988, with M4. (One year later I had my own little AT&T 3B2/400 running UNIX System V with SCCS!) Having now finally read more about M6 thanks to your link to a copy of CSTR#2, I see that vc is moderately more like M6 than M4 in its simplicity, and as such it might be simpler to use in some contexts. -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: OpenPGP Digital Signature URL: From mrochkind at gmail.com Wed Dec 18 09:05:39 2024 From: mrochkind at gmail.com (Marc Rochkind) Date: Tue, 17 Dec 2024 16:05:39 -0700 Subject: [TUHS] SCCS, TeamWare, BitKeeper, and Git In-Reply-To: References: Message-ID: On Tue, Dec 17, 2024 at 3:38 PM Greg A. Woods wrote: > One thing I left unstated in my question, and indeed in my 1988 question > as well, was more or less: "If vc is just a macro processor, why might > I want to use it instead of any other available macro processor?" > Well, I don't remember much about why I designed vc. I was certainly very aware of language independent macro processors. I had read McIlroy's 1960 paper when I was still in college. ("Macro instruction extensions of compiler languages" CACM, Volume 3, Issue 4) And, I knew about m6, too. My guess is that there might have been two reasons for vc: That the other choices were too complicated for the programmers we were supporting, and that it was focused on excluding and including large blocks of text, which as I recall, were awkward with m6 (and maybe m4). In those days at Bell Labs we didn't think too hard about whether a new command should be added to the system. Ironically, this was true even though the UNIX kernel was a minimalist system and was as famous for what it left out as for what it was in it. No management approval was necessary for a new command; we just put it into /usr/bin on our PWB system (not the research system) along with a man page. Although we were in NJ, it was pretty much the wild west. Then, a few years later, the PWB system was packaged up and released into the world. Marc -------------- next part -------------- An HTML attachment was scrubbed... URL: