From roland.mainz at nrubsig.org Sun Mar 2 23:40:26 2008 From: roland.mainz at nrubsig.org (Roland Mainz) Date: Mon, 03 Mar 2008 08:40:26 +0100 Subject: [busybox-dev] ksh93-integration 2008-02-29 update binaries available for download (for Nevada >= B72+Indiana) ... Message-ID: <47CBAB6A.29243300@nrubsig.org> Hi! ---- 2008-02-29: ksh93 update tarballs for OpenSolaris These tarballs are intended to be installed over an existing OpenSolaris >= B72 + Indiana distribution i386 or SPARC installation and provide ksh93s+_20080202 (the content matches PSARC/2008/094 (http://www.opensolaris.org/os/community/arc/caselog/2008/094/)). Note that the binaries are provided for testing and evaluation purpose ONLY. Please report any problems/errors/bugs/comments to the ksh93-integration project bugzilla[5] or the ksh93-integration mailinglist[4] (please subscribe before posting). ** Highlights of this release: - "multiline" editor mode allows to edit lines in a similar fashion as "readline"-based applications like "bash" - /etc/ksh.kshrc now sets a default prompt for interactive sessions (see NOTES below) - Many bugfixes - Many closed-source binaries are replaced with opensource versions - The shell script compiler "shcomp" is now included - Many new demos including: - "shnote" (utilty to read/write pastebin.ca) - "shtwitter" (access twitter.com service and push blog entries) - "shpiano" (audio demo) - "shtinyurl" (tool to generate short tinyurl.com URLs) - "shman" (early prototype of a /usr/bin/man rewrite) - "svcproptree1" (SMF demo) - "xmldocumenttree1" (XML parser demo) - "test_net_sctp" (SCTP demo) - "multifollow" (poll(1) demo which run multiple "tail -f") ... and much more... ** Install instructions: 1. Download the tarball: + i386/AMD64: http://www.opensolaris.org/os/project/ksh93-integration/downloads/ksh93_integration_20080229_snapshot_i386.tar.bz2 + SPARC: http://www.opensolaris.org/os/project/ksh93-integration/downloads/ksh93_integration_20080229_snapshot_sparc.tar.bz2 2. Verify the MD5 checksum: + i386/AMD64: MD5 (ksh93_integration_20080229_snapshot_i386.tar.bz2)= 7a1bae17aeeccd62df31a1c5d8295554 + SPARC: MD5 (ksh93_integration_20080229_snapshot_sparc.tar.bz2)= 45c11e457118232f7fd1d07b0f178b1b 3. Login as user "root": 4. Change directory to / and unpack the tarball with /usr/bin/tar using the "xvof" option ("o" is very important to set the file ownership to "root") * Example for i386/AMD64: $ cd /tmp $ wget http://www.opensolaris.org/os/project/ksh93-integration/downloads/ksh93_integration_20080229_snapshot_i386.tar.bz2 $ /usr/sfw/bin/openssl md5 ksh93_integration_20080229_snapshot_i386.tar.bz2 MD5(ksh93_integration_20080229_snapshot_i386.tar.bz2)= 7a1bae17aeeccd62df31a1c5d8295554 # cd / # sync ; sync # bzcat @:$ for normal users and @:# for user "root"; the prompt length itself is limited to ~~20 characters to ensure it only occupies 1/4 of a standard 80x24 terminal window. * was added to emacs/gmacs mode to clear the screen (per community requests and to be in sync with bash) * 64bit binaries and libraries are now included (and used by default if the hardware is 64bit capable) * "shcomp", the shell script compiler is now included as /usr/bin/shcomp. * AST l10n utilities are stored in /usr/ast/bin/. * Starting with ksh93s+ multibyte characters can be used for variable/function/etc.-names. Please test this functionality extensively. * Users of B72, B73, B74 and B75 (anything ?B75 has already been fixed) which like to use ksh93 as login shell must create the /etc/shells (see shells(4) manual page) to include it in the list of "allowed" system login shells (otherwise dtlogin will refuse graphical login): Example /etc/shells file (created using $ cat usr/src/lib/libc/port/gen/getusershell.c | egrep '.*".*/(bin|sbin)/.*".*' | sed 's/[",]//g' | sort -u #): /bin/bash /bin/csh /bin/jsh /bin/ksh /bin/ksh93 /bin/pfcsh /bin/pfksh /bin/pfsh /bin/sh /bin/tcsh /bin/zsh /sbin/jsh /sbin/pfsh /sbin/sh /usr/bin/bash /usr/bin/csh /usr/bin/jsh /usr/bin/ksh /usr/bin/ksh93 /usr/bin/pfcsh /usr/bin/pfksh /usr/bin/pfsh /usr/bin/sh /usr/bin/tcsh /usr/bin/zsh /usr/sfw/bin/zsh /usr/xpg4/bin/sh * The tarball was created using the build_ksh93_standalone_tarball.sh script which is available in the usr/-directory of the prototype010 tree. Note: The script can only be used after a successfull $ make setup ; dmake install #-sequence, otherwise the resulting tarball will be incomplete. * The tarballs do not provide a manual page for ksh93. Please use the manual page for ksh93s+ in the meantime. * The ksh93 binaries can be build from source like this: (Instructions are for Solaris i386/AMD64; SPARC requires minor adjustments) 1. Pull sources and extract closed bin stuff (files can be obtained from opensolaris.org): $ mkdir test_x86 ; cd test_x86 $ svn checkout -r 957 svn://svn.genunix.org/on/branches/ ksh93/gisburn/prototype010/usr $ bzcat <../download/on-closed-bins-nd-b72.i386.tar.bz2 | tar -xf - $ cd .. 2. Create opensolaris.sh. This is the usual opensolaris.sh with the paths adjusted to match your location of the sources. Example for the changes applies to opensolaris.sh (for my workspace): --- ./test1_x86/usr/src/tools/env/opensolaris.sh Thu Jan 666 13:17:59 2666 +++ ./opensolaris.sh Sun Jul 30 00:50:08 2666 @@ -43,10 +43,10 @@ # This is a variable for the rest of the script - GATE doesn't matter to # nightly itself -GATE=testws; export GATE +GATE=test1_x86; export GATE # CODEMGR_WS - where is your workspace at (or what should nightly name it) -CODEMGR_WS="/export/$GATE"; export CODEMGR_WS +CODEMGR_WS="/home/test001/ksh93/on_build1/$GATE"; export CODEMGR_WS # Location of encumbered binaries. ON_CLOSED_BINS="$CODEMGR_WS/closed"; export ON_CLOSED_BINS 3. Run "bldenv": $ env - SHELL=$SHELL TERM=$TERM HOME=$HOME LOGNAME=$LOGNAME DISPLAY=$DISPLAY XAUTHORITY=$XAUTHORITY LANG=C LC_ALL=C PAGER=less MANPATH=$MANPATH /opt/onbld/bin/bldenv opensolaris.sh # 4. Build it (the quick way): $ cd test_x86/usr/src $ export CW_NO_SHADOW=1 $ time nice make setup 2>&1 | tee -a buildlog_setup.log $ time nice dmake install >buildlog.log 2>&1 Finally: Please check http://www.opensolaris.org/os/project/ksh93-integration/downloads/2008-02-29/ for any updates or additional comments... ** Links/References: [1]=ksh93-integration/migration project home page: http://www.opensolaris.org/os/project/ksh93-integration/ [2]=http://www.opensolaris.org/os/project/ksh93-integration/downloads/2008-02-29/ [3]=ksh93s+ manual page: http://www.opensolaris.org/os/project/ksh93-integration/docs/ksh93s/man/man1/sh/ [4]=ksh93-integration mailinglist: http://mail.opensolaris.org/mailman/listinfo/ksh93-integration-discuss/ ; please subscribe before posting (and please avoid flamewars) !! [5]=ksh93-integration project bugzilla: http://bugs.grommit.com/enter_bug.cgi?product=ksh93-integration [6]=Heads-up message for "ksh93 integrated into Solaris 11/B72": http://www.opensolaris.org/os/community/on/flag-days/pages/2007081701/ ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz at nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;) From roland.mainz at nrubsig.org Mon Mar 3 17:42:03 2008 From: roland.mainz at nrubsig.org (Roland Mainz) Date: Tue, 04 Mar 2008 02:42:03 +0100 Subject: [busybox-dev] ksh93/busybox (student) ToDo list... / was: Re: [ksh93-integration-discuss] need help to buildksh93 References: <209493.43462.qm@web7913.mail.in.yahoo.com> <47AA476C.8FAD7E95@nrubsig.org> Message-ID: <47CCA8EB.CE7D0CDA@nrubsig.org> Josh Hurst wrote: > On Feb 7, 2008 12:49 AM, Roland Mainz wrote: > > chinmayi sk wrote: [snip] > I think chinmayi also needs a list of the specific tasks for his students Below are some tasks/ideas what still needs to be done (for both ksh93-integration and the busybox project): - Write a Dtrace script which lists all executables with their paths which are used at boot time including their execution time and execution count (e.g. how often the command was used). This should help identify canidates of commands which should be supplied as builtins either via libcmd.so.1 or libbusybox.so.1 - Implement Dtrace support for ksh93 (although I'm unhappy with the Dtrace implementation in the Bourne shell because it leaves-out most of the important shell stuff... but that's another long email... ;-( ) - Port the libast regex functions (which have a set of additional features, including that regex functions can any other (supported) pattern matching format like shell/perl/literal patterns etc.) to Solaris libc (this is quite easy but some time, AFAIK this needs ~~ six manmonths (e.g. one engineer may need up to six months, two need three months etc.)) - Write a small library (libcompoundvar.so.1) which can read and write data in the compound variable format. This may be usefull to extend commands like /usr/bin/ls or the SMF tools to output the compound variable format that they can be read by the shell more easily without resorting to hand-crafted parsers for XML - Translate demo scripts to other language like Chinese, Japanese etc. (note that this is not about "localisation" (=l10n), the idea is to translate the variable+function names and comments to another language (to demostrate ksh93's feature to handle variable+function names etc. outside the ASCII range (see Sun Workshop/Forte/Studio's "-xcsi" option why this is a good idea))) - Remove depreciated interfaces from libast. Glenn Fowler has AFAIK a list of interfaces which are depreciated since some time but couldn't be removed yet because something else in the ast-open codebase (see http://www.research.att.com/~gsf/cgi-bin/download.cgi?action=list&name=ast-open) still depends on it (this is mainly work for upstream but would help reducing the size of libast a bit) - Repair profile shell code (e.g. pfksh93), see CR #6605478 ("ksh93 profile shell option does not work") - Update the TCL/TK parts of "tksh" to TCL/TK 8.5 - Do some profiling and figure out whether there is a way to improve the startup time of $ ksh93 -c 'false ; true' # ... - Write some tutorials, e.g. ... - "How to handle data in lists and trees in ksh93" - "Using datatypes in ksh93 (e.g. integer, float, string, compound datatypes)" - "Writing secure setuid/setgid scripts" ... more ideas for tutorials welcome... AFAIK that's all for now... ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz at nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;)