Re: [jed-users] new_process() can't handle input file > 65536 bytes?

[Roland Hughes <roland@xxxxxxxxxxxxxxxxxxxx>]

Wow! Too much time on hands!

In seriousness though, are you able to open the file in jed directly? I 
assume so, but your report didn't say.

Given that much of S-Lang is written in C and given lots of old code 
would malloc() 64K buffers for work areas because the High Memory Area 
of DOS (and other x86 operating systems) allowed for 64K 
paging/swapping. All part of IBM's infamous quote "640K ought to be 
enough for anybody."

Keep in mind it is Sunday and I haven't looked at any of the code, but 
I'm old, have created many of these legacy issues back in the day for 
other systems . . . because I had to . . . and 64K ought to be enough 
for anybody, right?

With C++ we have a host of String and Vector classes that can grow and 
shrink easily because they do not need contiguous storage. C was created 
to be a portable Assembly language though. It doesn't give us suck 
niceties. C operates closer to the metal. We could never malloc() more 
than 64K because with SEGMENT:OFFSET addressing, you could never get 
more than 64K in a single chunk. The smaller memory model tricks of 
"just increment the OFFSET" failed whenever your chunk crossed a SEGMENT 
boundary  and infinite loops happened trying to increment OFFSET in a 
loop trying to reach SEGMENT+1:OFFSET  because incrementing OFFSET would 
not change SEGMENT. It would just got to all FFFF then 0.

Yes, today we have different (but still not Motorola true linear memory) 
addressing. It works well for Page Swapping, and other such things. Most 
OS's have a Page (or pagelet) size of 512 bytes because that was the 
original disk block size. Your Page and Swap files needed to swap whole 
blocks. Every OS still needs a "base address" for each process so every 
process can be dynamically locatable at runtime.

The ham-fisted hack would be to hard code something way more than 64K 
for the buffer workspace. This would fail on either lesser or busier 
platforms where a contiguous > 64K chunk could not be found. You may 
have 2GB free on your system, but it can be busted up in such a way a 
>64K contiguous chunk of free RAM does not exist.

Here's a nice one to consider also. Depending on the OS, the user 
account can have a per-process memory limit. Their total over all memory 
access may be unlimited but each individual process has a hard cap. This 
exists to stop a single bad process from consuming all physical RAM 
causing a crash.

In the world of C++ a String class would start out small and keep 
increasing in size as needed until some kind of memory limit was 
encountered. The way iterators work it does not need to be contiguous.

Here are some discussions about string libraries for C

https://handmade.network/forums/t/3451-decent_string_lib_for_c

https://news.ycombinator.com/item?id=21692400

http://www.kigen.de/development/c/strings/index.html

https://github.com/anBertoli/simple-strings

https://bstring.sourceforge.net/

To do what you want, some form of dynamic string library has to be 
implemented/utilized for EVALBUFFER. Perhaps used internally for all of 
S-Lang, I do not know.

I have not played with any of these libraries because most of my 
embedded systems development has been C++. Been many years since I 
programmed in "just C."

On 6/29/2025 6:59 AM, Morten Bo Johansen wrote:
> Hi,
>
> If you have the time, then create a file, "t", of 65537 bytes
>
>     $ for i in {1..65537}; do echo -n a >> t; done
>
> In a *scratch* buffer, insert the following lines:
>
>     require("process");
>     variable obj = new_process(["cat"]; write=1, stdin="t");
>     () = obj.wait();
>
> and then:
>
>     alt-x evalbuffer
>     
> In my case the process hangs/freezes.
>
> If I remove one single 'a' from the file "t" to make the size 65536 bytes,
> then it completes.
>
> Is it a case of some integer data type somewhere in the slang source files
> that should be changed or is it just me who don't know what I'm doing?
>
> Regards,
> Morten
> _______________________________________________
> For list information, visit <http://jedsoft.org/jed/mailinglists.html>.
--
Roland Hughes, President
Logikal Solutions
(630)-205-1593  (cell)
https://theminimumyouneedtoknow.com
https://infiniteexposure.net
https://johnsmith-book.com

_______________________________________________
For list information, visit <http://jedsoft.org/jed/mailinglists.html>.