From roland.mainz at nrubsig.org Mon Jul 2 14:28:52 2007 From: roland.mainz at nrubsig.org (Roland Mainz) Date: Mon, 02 Jul 2007 23:28:52 +0200 Subject: [busybox-dev] Getting started with "busybox" ... References: <462EA8EB.CB2CFB4D@nrubsig.org> <9625978a0706121118q3d99557amf0cb88b1f2f0e6ad@mail.gmail.com> <466F9CF8.600E9A80@nrubsig.org> <319ee2b10706130729g4c949a38s654f55dfc67dcf2@mail.gmail.com> Message-ID: <46896E14.244639F9@nrubsig.org> G N S wrote: > On 6/13/07, Roland Mainz wrote: > > ... Moinak/GNS: Could you point me to a standalone command which you'd > > like to see converted (that I can create some sample source), please ? > > > > You may use one of the already implemented ones: > basename,cat,chgrp,chown,cut,date,dirname,echo,head,mkdir,rmdir,pathchk,pwd,rm,sleep,tail,wc,... > > Or not implemented ones: > chroot,env,grep,hostname,link,ls,touch,who,... Attached is a small patch ("ksh93_busybox_env_grep_patch001.diff.txt") which implements "env" and "grep"... "chroot", "hostname" and "touch" will follow later. Notes for the patch: - "grep" was taken from the ast-open distribution and only slightly modified, e.g. I removed all global variables and stored them in a structure allocated on the stack. I've used the AST "grep" since it support multibyte characters propery, if any path bindings are needed bind the matching builtins ("grep", "egrep", "fgrep" and "pgrep" (="pgrep" means "perl grep")) to /usr/xpg4/bin/ $ builtin -f libshell.so.1 egrep # $ builtin -f libshell.so.1 fgrep # $ builtin -f libshell.so.1 grep # $ builtin -f libshell.so.1 prep # - "env" was slightly more tricky because it more or less "forks" the environment tree, clears and/or sets variables and then prints the env variables or executes a command. I've used quick&&dirty implementation using |fork()|, however I am unhappy with this since it creates an (IMO unneccesary) process child instead of creating a subshell, modify the variable tree there and then use |posix_spawn()| to execute the command (the "env" builtin sets in libshell since I used a couple of basic variable manipulation functions) Usage: $ builtin -f libshell.so.1 env # More commands will follow later... ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz at nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;) -------------- next part -------------- Index: src/lib/libshell/common/bltins/env.c =================================================================== --- src/lib/libshell/common/bltins/env.c (revision 0) +++ src/lib/libshell/common/bltins/env.c (revision 0) @@ -0,0 +1,148 @@ + +#include +#include +#include +#include + +#include "defs.h" + +static +void putenv(Shell_t *shp, const char *name) +{ + register Namval_t *np; + if(name) + { + np = nv_open(name,shp->var_tree,NV_EXPORT|NV_IDENT|NV_NOARRAY|NV_ASSIGN); + if(!strchr(name,'=')) + nv_unset(np); + } +} + + +static const char usage[] = +"[-?\n@(#)$Id: env (AT&T Research) 2007-07-02 $\n]" +USAGE_LICENSE +"[+NAME?env - environment for command invocation]" +"[+DESCRIPTION?The env utility obtains the current environment, modifies it " + "according to its arguments, then invokes the utility named " + "by the utility operand with the modified environment.]" +"[i|-:ignore?Ignore environment.]" + "\n\n" +"[name=value] ... [utility [arg... ]]\n" +"\n" +"[+EXIT STATUS?]{" + "[+0?Successful Completion.]" + "[+>0?An error occurred.]" +"}" +"[+SEE ALSO?\bdirname\b(1), \bgetconf\b(1), \bbasename\b(3)]" +; + + +int +b_env(int argc, char **argv, void *context) +{ + Shell_t *shp = (Shell_t *)context; + char **p; + int opt; + int i; + int n; + const char *progname = argv[0]; + int nullenv = FALSE; + pid_t child; + + /* check for non-standard "-" option */ + if ((argc > 1) && (strcmp(argv[1], "-")) == 0) { + nullenv = TRUE; + for (i = 1; i < argc; i++) + argv[i] = argv[i+1]; + argc--; + } + + while (n = optget(argv, usage)) switch (n) + { + case 'i': + nullenv = TRUE; + break; + case ':': + error(2, "%s", opt_info.arg); + break; + case '?': + error(ERROR_usage(2), "%s", opt_info.arg); + break; + } + argv += opt_info.index; + argc -= opt_info.index; + if(error_info.errors) + error(ERROR_usage(2), "%s", optusage(NiL)); + + /* sync stdout/stderr to avoid that any outstanding data trigger + * problems with the |fork()|'ed child*/ + sfsync(sfstdout); + sfsync(sfstderr); + + /* This is inefficient. The code should only create a subshell and + * change the subshell environment and launch the command from this + * subshell (this would save the |fork()| below and would allow + * launching shell builtins) */ + if ((child = fork()) == (pid_t)0) + { + char **lenviron; + int exit_code = 0; + + if (nullenv) + { + char *s; + char buff[PATH_MAX*2+1]; + + lenviron = sh_envgen(); + while(s = *lenviron++) + { + strcpy(buff, s); /* buffer overflow ahead */ + if(s = strchr(buff,'=')) + { + *s='\0'; + } + + putenv(shp, buff); + } + } + + /* environment strings */ + while (argv[0] != NULL && strchr(argv[0], '=') != NULL) { + putenv(shp, argv[0]); + argv++; + } + + lenviron = sh_envgen(); + + /* if no utility, output environment strings */ + if (argv[0] == NULL) { + p = lenviron; + while (*p != NULL) + (void) puts(*p++); + } else { + + + (void) execve(argv[0], &argv[0], lenviron); + perror("execvp failed"); + exit_code = ((errno == ENOENT) || (errno == ENOTDIR)) ? 127 : 126; + } + + sfsync(sfstdout); + sfsync(sfstderr); + _exit(exit_code); + } + else + { + int wstat; + + if(child == (pid_t)-1) + error(ERROR_system(1), "cannot fork() child"); + /* Fixme: We shouldn't use |waitpid()| here (see above - + * subshell is better and won't require a |waitpid()|) */ + waitpid(child, &wstat, 0); + + return wstat; + } + /* not reached */ +} Index: src/lib/libshell/Makefile.com =================================================================== --- src/lib/libshell/Makefile.com (revision 718) +++ src/lib/libshell/Makefile.com (working copy) @@ -32,6 +32,7 @@ bltins/alarm.o \ bltins/cd_pwd.o \ bltins/cflow.o \ + bltins/env.o \ bltins/getopts.o \ bltins/hist.o \ bltins/misc.o \ Index: src/lib/libshell/mapfile-vers =================================================================== --- src/lib/libshell/mapfile-vers (revision 700) +++ src/lib/libshell/mapfile-vers (working copy) @@ -132,6 +132,7 @@ b_command; b_dot_cmd; b_dup; + b_env; b_eval; b_exec; b_false; Index: src/lib/libcmd/common/grep.c =================================================================== --- src/lib/libcmd/common/grep.c (revision 0) +++ src/lib/libcmd/common/grep.c (revision 0) @@ -0,0 +1,871 @@ +/*********************************************************************** +* * +* This software is part of the ast package * +* Copyright (c) 1995-2007 AT&T Knowledge Ventures * +* and is licensed under the * +* Common Public License, Version 1.0 * +* by AT&T Knowledge Ventures * +* * +* A copy of the License is available at * +* http://www.opensource.org/licenses/cpl1.0.txt * +* (with md5 checksum 059e8cd6165cb4c31e351f2b69388fd9) * +* * +* Information and Software Systems Research * +* AT&T Research * +* Florham Park NJ * +* * +* Glenn Fowler * +* * +***********************************************************************/ +#pragma prototyped + +static const char usage[] = +"[-?\n@(#)$Id: grep (AT&T Research) 2006-06-14 $\n]" +USAGE_LICENSE +"[+NAME?grep - search lines in files for matching patterns]" +"[+DESCRIPTION?The \bgrep\b commands search the named input files" +" for lines containing a match for the given \apatterns\a." +" Matching lines are printed by default. The standard input is searched" +" if no files are given or when the file \b-\b is specified.]" +"[+?There are six variants of \bgrep\b, each one using a different form of" +" \apattern\a, controlled either by option or the command path" +" base name. Details of each variant may be found in \bregex\b(3).]" +" {" +" [+grep?The default basic regular expressions (no alternations.)]" +" [+egrep?Extended regular expressions (alternations, one or more.)]" +" [+pgrep?\bperl\b(1) regular expressions (lenient extended.)]" +" [+xgrep?Augmented regular expressions (conjunction, negation.)]" +" [+fgrep?Fixed string expressions.]" +" [+agrep?Approximate regular expressions (not implemented.)]" +" }" +"[G:basic-regexp?\bgrep\b mode (default): basic regular expression \apatterns\a.]" +"[E:extended-regexp?\begrep\b mode: extended regular expression \apatterns\a.]" +"[X:augmented-regexp?\bxgrep\b mode: augmented regular expression \apatterns\a.]" +"[P:perl-regexp?\bpgrep\b mode: \bperl\b(1) regular expression \apatterns\a.]" +"[F:fixed-string?\bfgrep\b mode: fixed string \apatterns\a.]" +"[A:approximate-regexp?\bagrep\b mode: approximate regular expression \apatterns\a (not implemented.)]" + +"[C:context?Set the matched line context \abefore\a and \aafter\a count." +" By default only matched lines are printed.]:?" +" [before[,after]]:=2,2]" +"[c:count?Only print a matching line count for each file.]" +"[e:expression|pattern|regexp?Specify a matching \apattern\a. More than one" +" \apattern\a implies alternation. If this option is specified" +" then the command line \apattern\a must be omitted.]:" +" [pattern]" +"[f:file?Each line in \apattern-file\a is a \apattern\a, placed into a single" +" alternating expression.]:" +" [pattern-file]" +"[H:filename|with-filename?Prefix each matched line with the containing file name.]" +"[h:no-filename?Suppress containing file name prefix for each matched line.]" +"[i:ignore-case?Ignore case when matching.]" +"[l:files-with-matches?Only print file names with at least one match.]" +"[L:files-without-matches?Only print file names with no matches.]" +"[b:highlight?Highlight matches using the ansi terminal bold sequence.]" +"[v:invert-match|revert-match?Invert the \apattern\a match sense.]" +"[m:label?All patterns must be of the form \alabel\a:\apattern\a. Match and" +" count output will be prefixed by the corresponding \alabel\a:.]" +"[O:lenient?Enable lenient \apattern\a interpretation. This is the default.]" +"[x:line-match|line-regexp?Force \apatterns\a to match complete lines.]" +"[n:number|line-number?Prefix each matched line with its line number.]" +"[N:name?Set the standard input file name prefix to" +" \aname\a.]:[name:=empty]" +"[q:quiet|silent?Do not print matching lines.]" +"[S:strict?Enable strict \apattern\a interpretation with diagnostics.]" +"[s:suppress|no-messages?Suppress error and warning messages.]" +"[t:total?Only print a single matching line count for all files.]" +"[T:test?Enable implementation specific tests.]:" +" [test]" +"[w:word-match|word-regexp?Force \apatterns\a to match complete words.]" +"[a?Ignored for GNU compatibility.]" +"\n" +"\n[ pattern ] [ file ... ]\n" +"\n" +"[+DIAGNOSTICS?Exit status 0 if matches were found, 1 if no matches were found," +" where \b-v\b invertes the exit status. Exit status 2 for other" +" errors that are accompanied by a message on the standard error.]" +"[+SEE ALSO?\bed\b(1), \bsed\b(1), \bperl\b(1), \bregex\b(3)]" +"[+CAVEATS?Some expressions of necessity require exponential space" +" and/or time.]" +"[+BUGS?Some expressions may use sub-optimal algorithms. For example," +" don't use this implementation to compute primes.]" +; + +#include +#include +#include +#include +#include + +#ifndef EISDIR +#define EISDIR (-1) +#endif + +/* + * snarfed from Doug McElroy's C++ version + * + * this grep is based on the Posix re package. + * unfortunately it has to have a nonstandard interface. + * 1. fgrep does not have usual operators. REG_LITERAL + * caters for this. + * 2. grep allows null expressions, hence REG_NULL. + * 3. it may be possible to combine the multiple + * patterns of grep into single patterns. important + * special cases are handled by regcomb(). + * 4. anchoring by -x has to be done separately from + * compilation (remember that fgrep has no ^ or $ operator), + * hence REG_LEFT|REG_RIGHT. (An honest, but slow alternative: + * run regexec with REG_NOSUB off and nmatch=1 and check + * whether the match is full length) + */ + +typedef struct Item_s /* list item - sue me for waste */ +{ + struct Item_s* next; /* next in list */ + regex_t re; /* compiled re */ + Sfulong_t hits; /* labeled pattern matches */ + Sfulong_t total; /* total hits */ + char string[1]; /* string value */ +} Item_t; + +typedef struct List_s /* generic list */ +{ + Item_t* head; /* list head */ + Item_t* tail; /* list tail */ +} List_t; + +typedef struct State_s /* program state */ +{ + struct + { + char* base; /* sfsetbuf buffer */ + size_t size; /* sfsetbuf size */ + int noshare; /* turn off SF_SHARE */ + } buffer; + + List_t file; /* pattern file list */ + List_t pattern; /* pattern list */ + List_t re; /* re list */ + + regmatch_t posvec[1]; /* match position vector */ + regmatch_t* pos; /* match position pointer */ + int posnum; /* number of match positions */ + + int any; /* if any pattern hit */ + int list; /* list files with hits */ + int notfound; /* some input file not found */ + int options; /* regex options */ + + Sfulong_t hits; /* total matched pattern count */ + + unsigned char byline; /* multiple pattern line by line*/ + unsigned char count; /* count number of hits */ + unsigned char label; /* all patterns labeled */ + unsigned char match; /* match sense */ + unsigned char query; /* return status but no output */ + unsigned char number; /* line numbers */ + unsigned char prefix; /* print file prefix */ + unsigned char suppress; /* no unopenable file messages */ + unsigned char words; /* word matches only */ +} State_s; + +static void +addre(State_s *state, List_t* p, char* s) +{ + int c; + char* b; + Item_t* x; + Sfio_t* t; + + b = s; + if (state->label) + { + if (!(s = strchr(s, ':'))) + error(3, "%s: label:pattern expected", b); + c = s - b; + s++; + } + else + c = 0; + if (!(x = newof(0, Item_t, 1, c))) + error(ERROR_SYSTEM|3, "out of space (pattern `%s')", b); + if (c) + memcpy(x->string, b, c); + if (state->words) + { + if (!(t = sfstropen())) + error(ERROR_SYSTEM|3, "out of space (word pattern `%s')", s); + if (!(state->options & REG_AUGMENTED)) + sfputc(t, '\\'); + sfputc(t, '<'); + sfputr(t, s, -1); + if (!(state->options & REG_AUGMENTED)) + sfputc(t, '\\'); + sfputc(t, '>'); + if (!(s = sfstruse(t))) + error(ERROR_SYSTEM|3, "out of space"); + } + else + t = 0; + if (c = regcomp(&x->re, s, state->options|REG_MULTIPLE)) + regfatal(&x->re, 3, c); + if (t) + sfstrclose(t); + if (!p->head) + { + p->head = p->tail = x; + if (state->number || !regrecord(&x->re)) + state->byline = 1; + } + else if (state->label || regcomb(&p->tail->re, &x->re)) + { + p->tail = p->tail->next = x; + if (!state->byline && (state->number || !state->label || !regrecord(&x->re))) + state->byline = 1; + } + else + free(x); +} + +static void +addstring(State_s *state, List_t* p, char* s) +{ + Item_t* x; + + if (!(x = newof(0, Item_t, 1, strlen(s)))) + error(ERROR_SYSTEM|3, "out of space (string `%s')", s); + strcpy(x->string, s); + if (p->head) + p->tail->next = x; + else + p->head = x; + p->tail = x; +} + +static void +compile(State_s *state) +{ + int line; + size_t n; + char* s; + char* t; + char* file; + Item_t* x; + Sfio_t* f; + + for (x = state->pattern.head; x; x = x->next) + addre(state, &state->re, x->string); + for (x = state->file.head; x; x = x->next) + { + s = x->string; + if (!(f = sfopen(NiL, s, "r"))) + error(ERROR_SYSTEM|4, "%s: cannot open", s); + else + { + file = error_info.file; + error_info.file = s; + line = error_info.line; + error_info.line = 0; + while (s = (char*)sfreserve(f, SF_UNBOUND, SF_LOCKR)) + { + if (!(n = sfvalue(f))) + break; + if (s[n - 1] != '\n') + { + for (t = s + n; t > s && *--t != '\n'; t--); + if (t == s) + { + sfread(f, s, 0); + break; + } + n = t - s + 1; + } + s[n - 1] = 0; + addre(state, &state->re, s); + s[n - 1] = '\n'; + sfread(f, s, n); + } + while ((s = sfgetr(f, '\n', 1)) || (s = sfgetr(f, '\n', -1))) + { + error_info.line++; + addre(state, &state->re, s); + } + error_info.file = file; + error_info.line = line; + sfclose(f); + } + } + if (!state->re.head) + error(3, "no pattern"); +} + +static void +highlight(Sfio_t* sp, const char* s, int n, int so, int eo) +{ + static const char bold[] = {CC_esc,'[','1','m'}; + static const char normal[] = {CC_esc,'[','0','m'}; + + sfwrite(sp, s, so); + sfwrite(sp, bold, sizeof(bold)); + sfwrite(sp, s + so, eo - so); + sfwrite(sp, normal, sizeof(normal)); + sfwrite(sp, s + eo, n - eo); +} + +typedef struct +{ + State_s *state; + Item_t *item; +} record_handle; + +static int +record(void* handle, const char* s, size_t len) +{ + record_handle *r_x = (record_handle *)handle; + State_s *state = r_x->state; + Item_t *item = r_x->item; + + item->hits++; + if (state->query || state->list) + return -1; + if (!state->count) + { + if (state->prefix) + sfprintf(sfstdout, "%s:", error_info.file); + if (state->label) + sfprintf(sfstdout, "%s:", item->string); + if (state->pos) + highlight(sfstdout, s, len + 1, state->pos[0].rm_so, state->pos[0].rm_eo); + else + sfwrite(sfstdout, s, len + 1); + } + return 0; +} + +static void +execute(State_s *state, Sfio_t* input, char* name) +{ + register char* s; + char* file; + Item_t* x; + size_t len; + int result; + int line; + + Sfulong_t hits = 0; + + if (state->buffer.noshare) + sfset(input, SF_SHARE, 0); + if (state->buffer.size) + sfsetbuf(input, state->buffer.base, state->buffer.size); + if (!name) + name = "/dev/stdin"; + file = error_info.file; + error_info.file = name; + line = error_info.line; + error_info.line = 0; + if (state->byline) + { + for (;;) + { + error_info.line++; + if (s = sfgetr(input, '\n', 0)) + len = sfvalue(input) - 1; + else if (s = sfgetr(input, '\n', -1)) + { + len = sfvalue(input); + s[len] = '\n'; +#if _you_like_the_noise + error(1, "newline appended"); +#endif + } + else + { + if (sferror(input) && errno != EISDIR) + error(ERROR_SYSTEM|2, "read error"); + break; + } + x = state->re.head; + do + { + if (!(result = regnexec(&x->re, s, len, state->posnum, state->pos, 0))) + { + if (!state->label) + break; + x->hits++; + if (state->query || state->list) + goto done; + if (!state->count) + { + if (state->prefix) + sfprintf(sfstdout, "%s:", name); + if (state->number) + sfprintf(sfstdout, "%d:", error_info.line); + sfprintf(sfstdout, "%s:", x->string); + if (state->pos) + highlight(sfstdout, s, len + 1, state->pos[0].rm_so, state->pos[0].rm_eo); + else + sfwrite(sfstdout, s, len + 1); + } + } + else if (result != REG_NOMATCH) + regfatal(&x->re, 3, result); + } while (x = x->next); + if (!state->label && (x != 0) == state->match) + { + hits++; + if (state->query || state->list) + break; + if (!state->count) + { + if (state->prefix) + sfprintf(sfstdout, "%s:", name); + if (state->number) + sfprintf(sfstdout, "%d:", error_info.line); + if (state->pos) + highlight(sfstdout, s, len + 1, state->pos[0].rm_so, state->pos[0].rm_eo); + else + sfwrite(sfstdout, s, len + 1); + } + } + } + } + else + { + register char* e; + register char* t; + char* r; + + static char* span = 0; + static size_t spansize = 0; + + s = e = 0; + for (;;) + { + if (s < e) + { + t = span; + for (;;) + { + len = 2 * (e - s) + t - span + 1; + len = roundof(len, SF_BUFSIZE); + if (spansize < len) + { + spansize = len; + len = t - span; + if (!(span = newof(span, char, spansize, 0))) + error(ERROR_SYSTEM|3, "%s: line longer than %lu characters", name, len + e - s); + t = span + len; + } + len = e - s; + memcpy(t, s, len); + t += len; + if (!(s = sfreserve(input, SF_UNBOUND, 0)) || (len = sfvalue(input)) <= 0) + { + if ((sfvalue(input) || sferror(input)) && errno != EISDIR) + error(ERROR_SYSTEM|2, "%s: read error", name); + break; + } + else if (!(e = memchr(s, '\n', len))) + e = s + len; + else + { + r = s + len; + len = (e - s) + t - span; + len = roundof(len, SF_BUFSIZE); + if (spansize < len) + { + spansize = len; + len = t - span; + if (!(span = newof(span, char, spansize, 0))) + error(ERROR_SYSTEM|3, "%s: line longer than %lu characters", name, len + e - s); + t = span + len; + } + len = e - s; + memcpy(t, s, len); + t += len; + s += len + 1; + e = r; + break; + } + } + *t = '\n'; + x = state->re.head; + do + { + record_handle r_x = { state, x }; + if ((result = regrexec(&x->re, span, t - span, state->posnum, state->pos, state->options, '\n', (void*)&r_x, record)) < 0) + goto done; + if (result && result != REG_NOMATCH) + regfatal(&x->re, 3, result); + } while (x = x->next); + if (!s) + break; + } + else + { + if (!(s = sfreserve(input, SF_UNBOUND, 0))) + { + if ((sfvalue(input) || sferror(input)) && errno != EISDIR) + error(ERROR_SYSTEM|2, "%s: read error", name); + break; + } + if ((len = sfvalue(input)) <= 0) + break; + e = s + len; + } + t = e; + while (t > s) + if (*--t == '\n') + { + x = state->re.head; + do + { + record_handle r_x = { state, x }; + if ((result = regrexec(&x->re, s, t - s, state->posnum, state->pos, state->options, '\n', (void*)&r_x, record)) < 0) + goto done; + if (result && result != REG_NOMATCH) + regfatal(&x->re, 3, result); + } while (x = x->next); + s = t + 1; + break; + } + } + } + done: + error_info.file = file; + error_info.line = line; + if (state->byline && !state->label) + { + if (hits && state->list >= 0) + state->any = 1; + if (!state->query) + { + if (!state->list) + { + if (state->count) + { + if (state->count & 2) + state->hits += hits; + else + { + if (state->prefix) + sfprintf(sfstdout, "%s:", name); + sfprintf(sfstdout, "%I*u\n", sizeof(hits), hits); + } + } + } + else if ((hits != 0) == (state->list > 0)) + { + if (state->list < 0) + state->any = 1; + sfprintf(sfstdout, "%s\n", name); + } + } + } + else + { + x = state->re.head; + do + { + if (x->hits && state->list >= 0) + { + state->any = 1; + if (state->query) + break; + } + if (!state->query) + { + if (!state->list) + { + if (state->count) + { + if (state->count & 2) + { + x->total += x->hits; + state->hits += x->hits; + } + else + { + if (state->prefix) + sfprintf(sfstdout, "%s:", name); + if (state->label) + sfprintf(sfstdout, "%s:", x->string); + sfprintf(sfstdout, "%I*u\n", sizeof(x->hits), x->hits); + } + } + } + else if ((x->hits != 0) == (state->list > 0)) + { + if (state->list < 0) + state->any = 1; + if (state->label) + sfprintf(sfstdout, "%s:%s\n", name, x->string); + else + sfprintf(sfstdout, "%s\n", name); + } + } + x->hits = 0; + } while (x = x->next); + } +} + + +static +int grep_main(int argc, char** argv, void *context) +{ + int c; + char* s; + char* h; + Sfio_t* f; + State_s state; + memset(&state, 0, sizeof(state)); + + NoP(argc); + state.match = 1; + state.options = REG_FIRST|REG_NOSUB|REG_NULL; + h = 0; + if (strcmp(astconf("CONFORMANCE", NiL, NiL), "standard")) + state.options |= REG_LENIENT; + if (s = strrchr(argv[0], '/')) + s++; + else + s = argv[0]; + switch (*s) + { + case 'e': + case 'E': + s = "egrep"; + state.options |= REG_EXTENDED; + break; + case 'f': + case 'F': + s = "fgrep"; + state.options |= REG_LITERAL; + break; + case 'p': + case 'P': + s = "pgrep"; + state.options |= REG_EXTENDED|REG_LENIENT; + break; + case 'x': + case 'X': + s = "xgrep"; + state.options |= REG_AUGMENTED; + break; + default: + s = "grep"; + break; + } + error_info.id = s; + while (c = optget(argv, usage)) + switch (c) + { + case 'E': + state.options |= REG_EXTENDED; + break; + case 'F': + state.options |= REG_LITERAL; + break; + case 'G': + state.options &= ~(REG_AUGMENTED|REG_EXTENDED); + break; + case 'H': + state.prefix = opt_info.num; + break; + case 'L': + state.list = -opt_info.num; + break; + case 'N': + h = opt_info.arg; + break; + case 'O': + state.options |= REG_LENIENT; + break; + case 'P': + state.options |= REG_EXTENDED|REG_LENIENT; + break; + case 'S': + state.options &= ~REG_LENIENT; + break; + case 'T': + s = opt_info.arg; + switch (*s) + { + case 'b': + case 'm': + c = *s++; + state.buffer.size = strton(s, &s, NiL, 1); + if (c == 'b' && !(state.buffer.base = newof(0, char, state.buffer.size, 0))) + error(ERROR_SYSTEM|3, "out of space [test buffer]"); + if (*s) + error(3, "%s: invalid characters after test", s); + break; + case 'f': + state.options |= REG_FIRST; + break; + case 'l': + state.options |= REG_LEFT; + break; + case 'n': + state.buffer.noshare = 1; + break; + case 'r': + state.options |= REG_RIGHT; + break; + default: + error(3, "%s: unknown test", s); + break; + } + break; + case 'X': + state.options |= REG_AUGMENTED; + break; + case 'a': + break; + case 'b': + state.options &= ~(REG_FIRST|REG_NOSUB); + break; + case 'c': + state.count |= 1; + break; + case 'e': + addstring(&state, &state.pattern, opt_info.arg); + break; + case 'f': + addstring(&state, &state.file, opt_info.arg); + break; + case 'h': + state.prefix = 2; + break; + case 'i': + state.options |= REG_ICASE; + break; + case 'l': + state.list = opt_info.num; + break; + case 'm': + state.label = 1; + break; + case 'n': + state.number = 1; + break; + case 'q': + state.query = 1; + break; + case 's': + state.suppress = opt_info.num; + break; + case 't': + state.count |= 2; + break; + case 'v': + if (state.match = !opt_info.num) + state.options &= ~REG_INVERT; + else + state.options |= REG_INVERT; + break; + case 'w': + state.words = 1; + break; + case 'x': + state.options |= REG_LEFT|REG_RIGHT; + break; + case '?': + error(ERROR_USAGE|4, "%s", opt_info.arg); + break; + case ':': + error(2, "%s", opt_info.arg); + break; + default: + error(3, "%s: not implemented", opt_info.name); + break; + } + argv += opt_info.index; + if ((state.options & REG_LITERAL) && (state.options & (REG_AUGMENTED|REG_EXTENDED))) + error(3, "-F and -A or -P or -X are incompatible"); + if ((state.options & REG_LITERAL) && state.words) + error(ERROR_SYSTEM|3, "-F and -w are incompatible"); + if (!state.file.head && !state.pattern.head) + { + if (!argv[0]) + error(3, "no pattern"); + addstring(&state, &state.pattern, *argv++); + } + if (!(state.options & (REG_FIRST|REG_NOSUB))) + { + if (state.count || state.list || state.query || (state.options & REG_INVERT)) + state.options |= REG_FIRST|REG_NOSUB; + else + { + state.pos = state.posvec; + state.posnum = elementsof(state.posvec); + } + } + compile(&state); + if (!argv[0]) + { + state.prefix = h ? 1 : 0; + execute(&state, sfstdin, h); + } + else + { + if (state.prefix > 1) + state.prefix = 0; + else if (argv[1]) + state.prefix = 1; + while (s = *argv++) + { + if (f = sfopen(NiL, s, "r")) + { + execute(&state, f, s); + sfclose(f); + if (state.query && state.any) + break; + } + else + { + state.notfound = 1; + if (!state.suppress) + error(ERROR_SYSTEM|2, "%s: cannot open", s); + } + } + } + if ((state.count & 2) && !state.query && !state.list) + { + if (state.label) + { + Item_t* x; + + x = state.re.head; + do + { + sfprintf(sfstdout, "%s:%I*u\n", x->string, sizeof(x->total), x->total); + } while (x = x->next); + } + else + sfprintf(sfstdout, "%I*u\n", sizeof(state.hits), state.hits); + } + return (state.notfound && !state.query) ? 2 : !state.any; +} + + +int b_egrep(int argc, char** argv, void *context) +{ + return grep_main(argc, argv, context); +} + +int b_grep(int argc, char** argv, void *context) +{ + return grep_main(argc, argv, context); +} + +int b_fgrep(int argc, char** argv, void *context) +{ + return grep_main(argc, argv, context); +} + +int b_pgrep(int argc, char** argv, void *context) +{ + return grep_main(argc, argv, context); +} Index: src/lib/libcmd/Makefile.com =================================================================== --- src/lib/libcmd/Makefile.com (revision 718) +++ src/lib/libcmd/Makefile.com (working copy) @@ -41,6 +41,7 @@ date.o \ dirname.o \ expr.o \ + grep.o \ fds.o \ fmt.o \ fold.o \ Index: src/lib/libcmd/mapfile-vers =================================================================== --- src/lib/libcmd/mapfile-vers (revision 694) +++ src/lib/libcmd/mapfile-vers (working copy) @@ -40,11 +40,14 @@ b_cut; b_date; b_dirname; + b_egrep; b_expr; b_fds; + b_fgrep; b_fmt; b_fold; b_getconf; + b_grep; b_head; b_id; b_join; @@ -55,6 +58,7 @@ b_mv; b_paste; b_pathchk; + b_pgrep; b_rev; b_rm; b_rmdir; From roland.mainz at nrubsig.org Mon Jul 2 14:44:44 2007 From: roland.mainz at nrubsig.org (Roland Mainz) Date: Mon, 02 Jul 2007 23:44:44 +0200 Subject: [busybox-dev] Name of busybox binary (and library) ? Message-ID: <468971CC.5B71D8F7@nrubsig.org> Hi! ---- Two (tiny) questions: 1. How should the application "linking" be done, e.g. ... ...should the physical file /usr/bin/grep be a shell script which points to the "busybox" command and execute a -- snip -- #!/usr/bin/ksh93 cmd="$(basename "$0")" builtin -f busybox "${cmd}" ${cmd} "$@" -- snip -- (this is how the current alias.sh links tools like /usr/bin/kill to the builtin commands) or... ...should we use explicit binaries to do the same job (e.g. the C code for /usr/bin/grep links to libbusybox.so.1 and it's |main()| directly calls |b_grep()| - this would be faster but may be much harder to maintain and will eat-up more disk space) ? 2. Should we create a seperate library instead of putting everything into libcmd (IMO better to avoid havong problems with syncing the main ksh93 sources with upstream) ? And if "yes" - which name should we use (I propose "libbusybox.so.1") ? ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz at nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;) From shivakumar.gn at gmail.com Thu Jul 5 00:03:02 2007 From: shivakumar.gn at gmail.com (S h i v .) Date: Thu, 5 Jul 2007 12:33:02 +0530 Subject: [busybox-dev] Name of busybox binary (and library) ? In-Reply-To: <468971CC.5B71D8F7@nrubsig.org> References: <468971CC.5B71D8F7@nrubsig.org> Message-ID: <319ee2b10707050003n2165c931o42f1efefce2d8e08@mail.gmail.com> On 7/3/07, Roland Mainz wrote: > > Hi! > > ---- > > Two (tiny) questions: > 1. How should the application "linking" be done, e.g. ... > > ...should the physical file /usr/bin/grep be a shell script which points > to the "busybox" command and execute a > -- snip -- > #!/usr/bin/ksh93 > cmd="$(basename "$0")" > builtin -f busybox "${cmd}" > ${cmd} "$@" > -- snip -- > (this is how the current alias.sh links tools like /usr/bin/kill to the > builtin commands) > > or... > > ...should we use explicit binaries to do the same job (e.g. the C code > for /usr/bin/grep links to libbusybox.so.1 and it's |main()| directly > calls |b_grep()| - this would be faster but may be much harder to > maintain and will eat-up more disk space) ? > Since the purpose is to integrate it into ksh93, the first option is better. Second option effectively means ksh93 doesn't have anything to do with clubbing the different commands into a libbusybox.so.1 ! > > 2. Should we create a seperate library instead of putting everything > into libcmd (IMO better to avoid havong problems with syncing the main > ksh93 sources with upstream) ? And if "yes" - which name should we use > (I propose "libbusybox.so.1") ? > +1 for both the points. Having a separate library is desirable. Keeps the upstream and the changes here independant of each other. The name libbusybox.so.1 is fine. regards Shiv From shivakumar.gn at gmail.com Thu Jul 5 00:06:30 2007 From: shivakumar.gn at gmail.com (S h i v .) Date: Thu, 5 Jul 2007 12:36:30 +0530 Subject: [busybox-dev] Getting started with "busybox" ... In-Reply-To: <46896E14.244639F9@nrubsig.org> References: <462EA8EB.CB2CFB4D@nrubsig.org> <9625978a0706121118q3d99557amf0cb88b1f2f0e6ad@mail.gmail.com> <466F9CF8.600E9A80@nrubsig.org> <319ee2b10706130729g4c949a38s654f55dfc67dcf2@mail.gmail.com> <46896E14.244639F9@nrubsig.org> Message-ID: <319ee2b10707050006h33ec6b7fwfb390bb788dfa850@mail.gmail.com> On 7/3/07, Roland Mainz wrote: > > Attached is a small patch ("ksh93_busybox_env_grep_patch001.diff.txt") > which implements "env" and "grep"... "chroot", "hostname" and "touch" > will follow later. > The input provided should be good enough. Will have a look. thanks Shiv From roland.mainz at nrubsig.org Sun Jul 8 20:24:14 2007 From: roland.mainz at nrubsig.org (Roland Mainz) Date: Mon, 09 Jul 2007 05:24:14 +0200 Subject: [busybox-dev] [appliances-discuss] [osol-announce] New OpenSolaris Project:busybox References: <10078660.1183946574985.JavaMail.Twebapp@oss-app1> Message-ID: <4691AA5E.29887B8E@nrubsig.org> Steven Herd wrote: > Is there any progress on this project? Yes (however it may be better to ask in busybox-dev at opensolaris.org) ... several things are going on in parallel: - the code review of the first ksh93-integration putback is currently underway (review ends at 13 Feb 2007 (hopefully not an unlucky day... :-) ))) - I've drafted up some primitive prototype code which shows how new builtin commands can be added to ksh93... but I still own Shiv a few more examples for commands (but didn't had time for that since real life and ksh93-intgration review is eating up lots of time) - some discussion is underway how to access builtin commands and whether we need a seperate frontend, how to name the backend library (e.g. "libbusybox.so.1") etc. ---- Bye, Roland P.S.: Setting Reply-To: to busybox-dev at opensolaris.org -- __ . . __ (o.\ \/ /.o) roland.mainz at nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;) From roland.mainz at nrubsig.org Sun Jul 8 20:42:16 2007 From: roland.mainz at nrubsig.org (Roland Mainz) Date: Mon, 09 Jul 2007 05:42:16 +0200 Subject: [busybox-dev] Name of busybox binary (and library) ? References: <468971CC.5B71D8F7@nrubsig.org> <319ee2b10707050003n2165c931o42f1efefce2d8e08@mail.gmail.com> Message-ID: <4691AE98.EC38ED30@nrubsig.org> "S h i v ." wrote: > On 7/3/07, Roland Mainz wrote: > > Two (tiny) questions: > > 1. How should the application "linking" be done, e.g. ... > > > > ...should the physical file /usr/bin/grep be a shell script which points > > to the "busybox" command and execute a > > -- snip -- > > #!/usr/bin/ksh93 > > cmd="$(basename "$0")" > > builtin -f busybox "${cmd}" > > ${cmd} "$@" > > -- snip -- > > (this is how the current alias.sh links tools like /usr/bin/kill to the > > builtin commands) > > > > or... > > > > ...should we use explicit binaries to do the same job (e.g. the C code > > for /usr/bin/grep links to libbusybox.so.1 and it's |main()| directly > > calls |b_grep()| - this would be faster but may be much harder to > > maintain and will eat-up more disk space) ? > > Since the purpose is to integrate it into ksh93, the first option is better. > Second option effectively means ksh93 doesn't have anything to do with > clubbing the different commands into a libbusybox.so.1 ! Erm, yes and no. Part of the idea of using builtins is that larger scripts can simply declare "builtin " at the beginning of the script and then use the builtin version (and avoid the whole |fork()|/|exec()|/|setlocale()|/etc.-overhead for each command). If you use external commands via specifying their full path (e.g. "/usr/bin/grep" instead of "grep") or not declaring them as builtin you still create a seperate process which costs lots of CPU time and extra memory. BTW: Another idea would be to create something like /usr/busybox/bin/ and add a seperate frontend binary called /usr/busybox/bin/sh (remember /usr/bin/ksh93 is only a frontend binary which directly calls into libshell.so.1, too) there which enables the libbusybox.so.1 builtins by default. For consumers /sbin/sh+/usr/bin/sh could then link to /usr/busybox/bin/sh and the alias.sh machinery wouldn't require an explicit "builtin -f busybox" for each command or all the system scripts. > > 2. Should we create a seperate library instead of putting everything > > into libcmd (IMO better to avoid havong problems with syncing the main > > ksh93 sources with upstream) ? And if "yes" - which name should we use > > (I propose "libbusybox.so.1") ? > > > > +1 for both the points. Having a separate library is desirable. Keeps > the upstream and the changes here independant of each other. > The name libbusybox.so.1 is fine. Ok... ... patch follows once I have cleared my tree from the current ksh93-integration review changes... ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz at nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;) From dgk at research.att.com Mon Jul 9 08:54:54 2007 From: dgk at research.att.com (David Korn) Date: Mon, 9 Jul 2007 11:54:54 -0400 Subject: [busybox-dev] Name of busybox binary (and library) ? Message-ID: <200707091554.l69FssMx025104@penguin.research.att.com> Subject: Re: [busybox-dev] Name of busybox binary (and library) ? -------- > Two (tiny) questions: > 1. How should the application "linking" be done, e.g. ... > > ...should the physical file /usr/bin/grep be a shell script which points > to the "busybox" command and execute a > -- snip -- > #!/usr/bin/ksh93 > cmd="$(basename "$0")" > builtin -f busybox "${cmd}" > ${cmd} "$@" > -- snip -- > (this is how the current alias.sh links tools like /usr/bin/kill to the > builtin commands) > > or... > > ...should we use explicit binaries to do the same job (e.g. the C code > for /usr/bin/grep links to libbusybox.so.1 and it's |main()| directly > calls |b_grep()| - this would be faster but may be much harder to > maintain and will eat-up more disk space) ? There is a third method which we use here. You could create a file named .paths in the bin directory where you want the command to exist and put the line BUILTIN_LIB=busybox in .paths. In this case whenever a command in that directory is also in the busybox library, the built-in version will be executed rather than the command. The .paths file also allows you to specify a FPATH location that will be used to look up functions associated with this directory. Thus, if you implement env as a function, then you can put the function in say ../fun and it will use this instead of the binary env program in this directory. > > > 2. Should we create a seperate library instead of putting everything > into libcmd (IMO better to avoid havong problems with syncing the main > ksh93 sources with upstream) ? And if "yes" - which name should we use > (I propose "libbusybox.so.1") ? This would be a good idea at this time. > > David Korn dgk at research.att.com From roland.mainz at nrubsig.org Fri Jul 13 19:15:50 2007 From: roland.mainz at nrubsig.org (Roland Mainz) Date: Sat, 14 Jul 2007 04:15:50 +0200 Subject: [busybox-dev] ksh93-integration 2007-07-14 test binaries available for download... Message-ID: <469831D6.C849B506@nrubsig.org> Hi! ---- A new set of tarballs containing an OS/Net version of ksh93 [1] (based on ksh93s+_beta_20070418 [2]) is now available from http://www.opensolaris.org/os/project/ksh93-integration/downloads/2007-07-14/ These tarballs are intended to be installed over an existing OpenSolaris i386 or SPARC installation (>= Nevada B61) and provide ksh93s+_beta_20070418 [2] for testing and evaluation purposed ONLY. Please report any problems/errors/bugs/comments to the ksh93-integration project bugzilla [5] or the ksh93-integration mailinglist [4]. ** Install instructions: 1. Download the tarball: + i386/AMD64: http://www.opensolaris.org/os/project/ksh93-integration/downloads/ksh93_integration_20070714_snapshot_i386.tar.bz2 + SPARC: http://www.opensolaris.org/os/project/ksh93-integration/downloads/ksh93_integration_20070714_snapshot_sparc.tar.bz2 2. Verify the MD5 checksum: + i386/AMD64: MD5 (ksh93_integration_20070714_snapshot_i386.tar.bz2)= cab10e4b10eebcf753da462a79bb69c9 + SPARC: MD5(ksh93_integration_20070714_snapshot_sparc.tar.bz2) = 003e62591c137cdd46e022baa6bc8f9c 3. Login as user "root": 4. Change directory to / and unpack the tarball with /usr/bin/tar using the "xvof" option ("o" is very important to set the file ownership to "root") Example for i386/AMD64: $ cd /tmp $ wget http://www.opensolaris.org/os/project/ksh93-integration/downloads/ksh93_integration_20070714_snapshot_i386.tar.bz2 $ /usr/sfw/bin/openssl md5 ksh93_integration_20070714_snapshot_i386.tar.bz2 MD5(ksh93_integration_20070714_snapshot_i386.tar.bz2)= cab10e4b10eebcf753da462a79bb69c9 # cd / # sync ; sync # bzcat was added to emacs/gmacs mode to clear the screen (per community requests and to be in sync with bash) * If you wish to use ksh93 as login shell you have to create the file /etc/shells (see shells(4) manual page) to include it in the list of "allowed" system login shells: Example /etc/shells file (created using $ cat usr/src/lib/libc/ port/gen/getusershell.c | egrep '.*".*/(bin|sbin)/.*".*' | sed 's /[",]//g' | sort -u #): /bin/bash /bin/csh /bin/jsh /bin/ksh /bin/ksh93 /bin/pfcsh /bin/pfksh /bin/pfsh /bin/sh /bin/tcsh /bin/zsh /sbin/jsh /sbin/pfsh /sbin/sh /usr/bin/bash /usr/bin/csh /usr/bin/jsh /usr/bin/ksh /usr/bin/ksh93 /usr/bin/pfcsh /usr/bin/pfksh /usr/bin/pfsh /usr/bin/sh /usr/bin/tcsh /usr/bin/zsh /usr/sfw/bin/zsh /usr/xpg4/bin/sh * libcmd.so is replaced with a version which includes both the ksh93 builtin commands and the private Solaris API of previous libcmd versions. * The tarball was simply created from an OS/Net B61 proto/ subdir via collecting the files listed by $ find $ROOT '!' -type d | sed 's/.*\/root_sparc\///' | egrep "/(lib|llib-l)(cmd|ast|shell|dll| pp)|/(ksh|rksh|pfksh)|include/ast|usr/ast/" | egrep -v cmdutils | sort #. * The tarballs do not provide a manual page for ksh93. Please use the manual page for ksh93r in the meantime. * "multiline" input mode was temporary disabled in the default configuration due small issues. * The ksh93 binaries can be build from source like this: (Instructions are for Solaris i386/AMD64; SPARC requires minor adjustments) 1. Pull sources and extract closed bin stuff (files can be obtained from opensolaris.org): $ mkdir test_x86 ; cd test_x86 $ svn checkout -r 740 svn://svn.genunix.org/on/branches/ksh93/gisburn/prototype005/usr $ bzcat <../download/on-closed-bins-nd-b61.i386.tar.bz2 | tar -xf - $ cd .. 2. Create opensolaris.sh. This is the usual opensolaris.sh with the paths adjusted to match your location of the sources. Example for the changes applies to opensolaris.sh (for my workspace): --- ./test1_x86/usr/src/tools/env/opensolaris.sh Thu Sep 14 13:17:59 2006 +++ ./opensolaris.sh Sun Jul 30 00:50:08 2006 @@ -43,10 +43,10 @@ # This is a variable for the rest of the script - GATE doesn't matter to # nightly itself -GATE=testws; export GATE +GATE=test1_x86; export GATE # CODEMGR_WS - where is your workspace at (or what should nightly name it) -CODEMGR_WS="/export/$GATE"; export CODEMGR_WS +CODEMGR_WS="/home/test001/ksh93/on_build1/$GATE"; export CODEMGR_WS # Location of encumbered binaries. ON_CLOSED_BINS="$CODEMGR_WS/closed"; export ON_CLOSED_BINS 3. Run "bldenv": $ env - SHELL=$SHELL TERM=$TERM HOME=$HOME LOGNAME=$LOGNAME DISPLAY=$DISPLAY XAUTHORITY=$XAUTHORITY LANG=C LC_ALL=C PAGER =less MANPATH=$MANPATH /opt/onbld/bin/bldenv opensolaris.sh # 4. Build it (the quick way): $ cd test_x86/usr/src $ export ON_BUILD_AST_L10N_CATALOGS=1 CW_NO_SHADOW=1 $ time nice make setup 2>&1 | tee -a buildlog_setup.log $ time nice dmake install >buildlog.log 2>&1 Finally: Please check http://www.opensolaris.org/os/project/ksh93-integration/downloads/2007-07-14/ for any updates or additional comments... ** Links/References: [1]=ksh93-integration/migration project home page: http://www.opensolaris.org/os/project/ksh93-integration/ [2]=ksh93s+_beta release annoucement: http://mail.opensolaris.org/pipermail/ksh93-integration-discuss/2007-April/002483.html [3]=ksh93r manual page: http://www.opensolaris.org/os/project/ksh93-integration/docs/ksh93r/man/man1/sh/ [4]=ksh93-integration mailinglist: http://mail.opensolaris.org/mailman/listinfo/ksh93-integration-discuss/ ; please subscribe before posting (and please avoid flamewars) !! [5]=http://bugs.grommit.com/enter_bug.cgi?product=ksh93-integration ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz at nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;)