
@@ 0,0 +1,543 @@



+



+* Table operations  filter or combine tables



+



+ This section within the library of babel provides table operations.



+ See the documentation just below for details and working examples.



+



+ Author : MarcOliver Ihm <ihm@ferntreffer.de>



+ Version : 1.0



+



+** Documentation



+



+*** Introduction



+



+ The table operations (currently four) are grouped in two categories:



+



+  Filtering the rows of a single table: keeping or removing



+  Combining two tables into one: merging or intersecting



+



+ All four operations are demonstrated below.



+



+*** Example tables



+



+ To demonstrate we need three tables: upper, lower and keys:



+



+#+name: upper



+ 1  A 



+ 3  C 



+ 4  D 



+ 10  J 



+ 2  B 



+



+#+name: lower



+ Position  Letter 



++



+ 2  b 



+ 4  d 



+ 5  e 



+ 6  h 



+



+#+name: keys



+ Position 



+



+ 1 



+ 2 



+ 4 



+



+ The tables upper and lower both have two columns and associate a position in



+ the alphabet with the matching letter. E.g. the row " 1  A " from table



+ upper, just states that the letter "A" comes at position 1 in the alphabet.



+



+ Nearly the same is true for table lower, only that it contains lower case



+ letters. Some of its letters (e.g. "b") have counterparts in table upper



+ ("B"), some (e.g. "e") dont.



+



+ The table keys finally, contains keys (i.e. positions within the alphabet),



+ that can be used to select rows from either table upper or lower.



+



+ Note, that tables may have column headings or not.



+



+*** Filtering a table



+



+**** Keeping rows



+



+ Let's say, we want to select the uppercase letters (i.e. rows from the



+ table upper), that are given in table keys (i.e. the first, second and



+ fourth letter).



+



+ This can be described as filtering table upper and keeping only those rows,



+ that appear in table keys.



+



+ As a babelcall, this reads:



+



+#+call: tableoperationsfilterkeep(upper,keys)



+



+#+results: tableoperationsfilterkeep(upper,keys)



+ 1  A 



+ 4  D 



+ 2  B 



+



+ ,which gives exactly those rows from table upper, that are specified in



+ keys.



+



+**** Removing rows



+



+ Now, if on the contrary you want to filter table upper to remove any rows,



+ which are given in table keys:



+



+#+call: tableoperationsfilterremove(upper,keys) :colnames yes



+



+#+results: tableoperationsfilterremove(upper,keys)



+ Position  t2c2 



++



+ 3  C 



+ 10  J 



+



+ ,which is the expected result.



+



+ Please note, that the call contains the header argument ":colnames yes",



+ which causes the result table to contain the headings "Position" and



+ "t2c2". These headings are taken from the inputtables upper and



+ keys. However, as upper does not contain any headings, the heading "t2c2"



+ is generated artificially; it stands for "table 2 column 2".



+



+ If you do not want to have column names in the result table, just leave out



+ the header argument ":colnames yes" like in the first example. Note



+ however, that ":colnames no" does not give the expected effect.



+



+*** Combining tables



+



+ Now, lets have a look at the tables upper and lower alone and see how to



+ combine them.



+



+ Note, that we only look at combining two tables for simplicity, however, all



+ operations can be easily scaled up to seven tables.



+



+**** Merging rows



+



+ We have two tables, one with upper case letters and one with lower



+ case. What now, if you want to have only one table, which contains both,



+ upper and lower case letters ?



+



+ You may want to merge them:



+



+#+call: tableoperationscombinemerge(upper,lower) :colnames yes



+



+#+results: tableoperationscombinemerge(upper,lower)



+ Position  t1c2  Letter 



+++



+ 1  A  



+ 2  B  b 



+ 3  C  



+ 4  D  d 



+ 5   e 



+ 6   h 



+ 10  J  



+



+



+ This result combines both upper and lower case letters and lists them by



+ their position within the alphabet.



+



+**** Intersecting rows



+



+ If you only want the rows, that are complete (i.e. have both upper and



+ lower case letters) you may compute the intersection:



+



+#+call: tableoperationscombineintersect(upper,lower)



+



+#+results: tableoperationscombineintersect(upper,lower)



+ 2  B  b 



+ 4  D  d 



+



+



+ ,which has only those keys and letters, that appear in both tables.



+



+ Note, that we have ommitted the headeragument ":colnames yes" so that the



+ result table has no headings.



+



+** Internals



+



+ This section is not required if you just want to use table operations as



+ described above. Only if you are curious about its implementation or



+ development, you might want to have a look.



+



+*** Implementation



+



+ Here is the actual lisp code, that implements the functionality of table



+ operations.



+



+**** tableoperationsfilter



+***** Directly callable blocks



+



+#+name: tableoperationsfilterkeep



+#+begin_src emacslisp :noweb yes :results silent :var table=() :var filter=()



+ <<lobtableoperationshelpergetheadingsdefun>>



+ <<lobtableoperationsfilterdefun>>



+ (let ((filterandtable (list filter table)))



+ (lobtableoperationsfilter 'keep filterandtable))



+#+end_src



+



+#+name: tableoperationsfilterremove



+#+begin_src emacslisp :noweb yes :results silent :var table=() :var filter=() :colnames nil



+ <<lobtableoperationshelpergetheadingsdefun>>



+ <<lobtableoperationsfilterdefun>>



+ (let ((filterandtable (list filter table)))



+ (lobtableoperationsfilter 'remove filterandtable))



+#+end_src



+



+***** Included defuns



+



+#+name: lobtableoperationsfilterdefun



+#+begin_src emacslisp



+ (defun lobtableoperationsfilter (what filterandtable)



+ "Internal function for table operations in orgmode library of babel"



+



+ (let (keys



+ resulttable



+ headingsalltables



+ filter



+ table)



+



+ ;; seperate headings from rest of tables



+ (setq headingsalltables



+ (lobtableoperationshelpergetheadings filterandtable))



+



+ ;; extract arguments



+ (setq filter (car filterandtable))



+ (setq table (cadr filterandtable))



+



+ ;; remove hlines



+ (setq table (orgbabeldelhlines table))



+ (setq filter (orgbabeldelhlines filter))



+ (setq keys (mapcar 'car filter))



+



+ ;; start result with headings (reversed)



+ (setq resulttable (cons 'hline (cons headingsalltables nil)))



+



+ (dolist (line table) ; loop over table lines



+ (if (equal (not (not (member (car line) keys)))



+ (equal what 'keep)) ; 'keep or 'remove ?



+ (setq resulttable (cons line resulttable))))



+ (nreverse resulttable)))



+#+end_src



+



+**** tableoperationscombine



+***** Directly callable blocks



+



+#+name: tableoperationscombinemerge



+#+begin_src emacslisp :noweb yes :results silent :var t1=() :var t2=() :var t3=() :var t4=() :var t5=() :var t6=() :var t7=()



+ <<lobtableoperationshelpergetheadingsdefun>>



+ <<lobtableoperationscombinedefun>>



+ (let ((tables (list t1 t2 t3 t4 t5 t6 t7)))



+ (lobtableoperationscombine 'merge tables))



+#+end_src



+



+#+name: tableoperationscombineintersect



+#+begin_src emacslisp :noweb yes :results silent :var t1=() :var t2=() :var t3=() :var t4=() :var t5=() :var t6=() :var t7=()



+ <<lobtableoperationshelpergetheadingsdefun>>



+ <<lobtableoperationscombinedefun>>



+ (let ((tables (list t1 t2 t3 t4 t5 t6 t7)))



+ (lobtableoperationscombine 'intersect tables))



+#+end_src



+



+***** Included defuns



+



+#+name: lobtableoperationscombinedefun



+#+begin_src emacslisp



+ (defun lobtableoperationscombine (what tables)



+ "Internal function for tableoperations in orgmode library of babel"



+ (let (isallnumbers



+ formatspecifier



+ restoftables



+ restsoftables



+ restofrestsoftables



+ restoftable



+ headingsalltables



+ widthsoftables



+ currentkey



+ currentkeyinintersection



+ resulttable



+ resultline



+ i)



+



+ ;; remove possible empty trailing tables



+ (setq restoftables tables)



+ (while (cadr restoftables) (setq restoftables (cdr restoftables)))



+ (setcdr restoftables nil)



+



+ ;; seperate headings from rest of tables



+ (setq headingsalltables (lobtableoperationshelpergetheadings



+ tables))



+ (setq resulttable (cons 'hline (cons headingsalltables nil)))



+



+ ;; remove all remaining hlines



+ (setq tables (mapcar 'orgbabeldelhlines tables))



+



+ ;; Find out, if all keys in all tables are numbers or if



+ ;; there are strings among them



+ (setq isallnumbers



+ (catch 'notanumber



+ (dolist (table tables)



+ (dolist (line table)



+ (unless (numberp (car line))



+ (throw 'notanumber 'nil))))



+ 't))



+



+ (setq formatspecifier (if isallnumbers "%g" "%s"))



+ ;; Prepare functions to treat table contents in a unified way



+ (flet ((convert (x)



+ (if isallnumbers



+ x



+ (if (numberp x)



+ (numbertostring x)



+ x)))



+ (lessthan (x y)



+ (if isallnumbers (< x y)



+ (string< (convert x)



+ (convert y))))



+ (compare (x y)



+ (if isallnumbers (= x y)



+ (string= (convert x)



+ (convert y)))))



+



+ ;; sort tables



+ (setq tables (mapcar (lambda (table)



+ (sort table (lambda (x y)



+ (lessthan (car x)



+ (car y)))))



+ tables))



+



+ ;; compute and remember table widths



+ (setq widthsoftables (mapcar (lambda (x) (length (car x))) tables))



+



+ ;; copy initially and shorten below



+ (setq restsoftables (copylist tables))



+



+ ;; loop as long as the rest of table still contains lines



+ (while (progn



+ ;; find lowest key among all tables, which is the key for the



+ ;; next line of the result



+ (setq currentkey nil)



+ (setq currentkeyinintersection 't) ; remember for later



+ (dolist (restoftable restsoftables) ; loop over all tables



+ (when (and restoftable ; and compare against all keys



+ (or (null currentkey)



+ (lessthan (caar restoftable)



+ currentkey)))



+ (setq currentkey (caar restoftable))))



+ currentkey)



+



+ (progn



+



+ (setq resultline (list currentkey))



+



+ ;; go through all tables and collect one line for the result table



+ (setq i 0) ; tablecount



+ ;; cannot use dolist like above, because we need to modify the



+ ;; conscells



+ (setq restofrestsoftables restsoftables)



+ (while (progn



+ (setq restoftable (car restofrestsoftables))



+ (incf i)



+ ;; if table contains current key



+ (if (and restoftable



+ (compare currentkey (caar restoftable)))



+ ;; then copy rest of line



+ (progn (nconc resultline (cdar restoftable))



+ ;; and shorten rest



+ (setcar restofrestsoftables



+ (cdar restofrestsoftables))



+ ;; and check, if currentkey appears again



+ (when (and (caadr restoftable)



+ (compare currentkey



+ (caadr restoftable)))



+ (error (concat "Key '"



+ formatspecifier



+ "' appears twice within "



+ "input table %i")



+ (convert currentkey) i)



+ )



+ )



+ ;; otherwise fill with nil and do not shorte



+ ;; rest of table



+ (progn



+ (setq currentkeyinintersection nil)



+ (nconc resultline (makelist (1



+ (elt widthsoftables



+ (1 i)))



+ ""))))



+



+ (setq restofrestsoftables



+ (cdr restofrestsoftables))



+ restofrestsoftables)) ; condition for loop



+ (if (or (eq what 'merge) currentkeyinintersection)



+ ;; store away line



+ (setq resulttable (cons



+ resultline



+ resulttable)))))



+



+ (nreverse resulttable))))



+#+end_src



+



+**** Common helper functions



+



+#+name: lobtableoperationshelpergetheadingsdefun



+#+begin_src emacslisp



+ (defun lobtableoperationshelpergetheadings (tables)



+ "Internal function for tableoperations in orgmode library of babel"



+ (let ((restoftables tables)



+ (i 1)



+ headingsalltables



+ headingsonetable



+ headingofkey)



+ (while restoftables



+ (progn



+ (setq table (car restoftables))



+ (if (eq (cadr table) 'hline)



+ ;; second line is a hline, so first is a heading



+ (progn



+ ; take headings from first table row



+ (setq headingsonetable (cdar table))



+ (unless headingofkey (setq headingofkey (caar table)))



+ (unless (string= headingofkey (caar table))



+ (error "Name of first column is not the same in all tables"))



+ (setcar restoftables



+ (cdar restoftables))) ; and shorten rest



+ ;; table does not contain headings, so make them up



+ (setq headingsonetable



+ (mapcar



+ (lambda (x) (format "t%dc%d" i x))



+ (numbersequence 2 (length (car table))))))



+ (setq headingsalltables (append headingsalltables



+ headingsonetable))



+ (setq restoftables (cdr restoftables))



+ (incf i)



+ restoftables)) ; condition for while loop



+ (unless headingofkey (setq headingofkey "key"))



+ (setq headingsalltables (cons headingofkey headingsalltables))



+ headingsalltables))



+



+#+end_src



+



+**** Debugging and testing



+***** Clean up



+#+begin_src emacslisp



+ (saveexcursion



+ (beginningofbuffer)



+ (while (researchforward "^#\\+results:.*\n\\(^\.+\n\\)*\n" nil t)



+ (replacematch ""))



+ )



+#+end_src



+



+#+results:



+



+***** Byte Compilation



+



+ (bytecompile 'lobtableoperationscombine)



+ (bytecompile 'lobtableoperationsfilter)



+



+*** Development



+**** Versions and history



+



+ [20120318 So] Version 1.0:



+  Added handling of hlines and table headings



+



+ [20120107 Sa] Version 0.01:



+  Restructured as a single orgfile; no special .elfile needed any more



+  Combined and restructured documentation and implementation



+



+**** Bugs and Todos



+



+  [X] Brush up documentation



+  [X] Stay below 80 columns



+  [X] Tests with more than two columns per table



+  [X] Tests with more than two tables for merging



+  [X] Handle optional table captions



+  [X] Handle hlines



+  [X] flet within lobtableoperationscombine



+  [] flet within directly callable blocks; try to avoid global functions



+ Not feasible, because that hinders debugging to much



+  [X] Use :results silent



+



+**** Testcases



+



+#+name: upperwide



+ Position  c1  c2  c3  c4 



+++++



+ 1  A1  A2  A3  A4 



+ 3  C1  C2  C3  C4 



+ 4  D1  D2  D3  D4 



+ 10  J1  J2  J3  J4 



+ 2  B1  B2  B3  B4 



+



+#+name: lowerwide



+ 2  b1  b2  b3  b4 



+ 4  d1  d2  d3  d4 



+ 5  e1  e2  e3  e4 



+ 6  h1  h2  h3  h4 



+



+#+name: upperlowerwide



+ 2  Bb1  Bb2  Bb3  Bb4 



+ 6  Hh1  Hh2  Hh3  Hh4 



+ 4  Dd1  Dd2  Dd3  Dd4 



+ 10  Jj1  Jj2  Jj3  Jj4 



+



+#+call: tableoperationsfilterkeep(upperwide,keys)



+



+#+results: tableoperationsfilterkeep(upperwide,keys)



+ 1  A1  A2  A3  A4 



+ 4  D1  D2  D3  D4 



+ 2  B1  B2  B3  B4 



+



+#+call: tableoperationsfilterremove(lowerwide,keys) :colnames yes



+



+#+results: tableoperationsfilterremove(lowerwide,keys)



+ Position  t2c2  t2c3  t2c4  t2c5 



+++++



+ 5  e1  e2  e3  e4 



+ 6  h1  h2  h3  h4 



+



+#+call: tableoperationscombinemerge(upperwide,lowerwide) :colnames yes



+



+#+results: tableoperationscombinemerge(upperwide,lowerwide)



+ Position  c1  c2  c3  c4  t2c2  t2c3  t2c4  t2c5 



+++++++++



+ 1  A1  A2  A3  A4     



+ 2  B1  B2  B3  B4  b1  b2  b3  b4 



+ 3  C1  C2  C3  C4     



+ 4  D1  D2  D3  D4  d1  d2  d3  d4 



+ 5      e1  e2  e3  e4 



+ 6      h1  h2  h3  h4 



+ 10  J1  J2  J3  J4     



+



+#+call: tableoperationscombineintersect(upperwide,lowerwide)



+



+#+results: tableoperationscombineintersect(upperwide,lowerwide)



+ 2  B1  B2  B3  B4  b1  b2  b3  b4 



+ 4  D1  D2  D3  D4  d1  d2  d3  d4 



+



+#+call: tableoperationscombinemerge(upperwide,lowerwide,upperlowerwide) :colnames yes



+



+#+results: tableoperationscombinemerge(upperwide,lowerwide,upperlowerwide)



+ Position  c1  c2  c3  c4  t2c2  t2c3  t2c4  t2c5  t3c2  t3c3  t3c4  t3c5 



+++++++++++++



+ 1  A1  A2  A3  A4         



+ 2  B1  B2  B3  B4  b1  b2  b3  b4  Bb1  Bb2  Bb3  Bb4 



+ 3  C1  C2  C3  C4         



+ 4  D1  D2  D3  D4  d1  d2  d3  d4  Dd1  Dd2  Dd3  Dd4 



+ 5      e1  e2  e3  e4     



+ 6      h1  h2  h3  h4  Hh1  Hh2  Hh3  Hh4 



+ 10  J1  J2  J3  J4      Jj1  Jj2  Jj3  Jj4 



+



+#+call: tableoperationscombineintersect(upperwide,lowerwide,upperlowerwide)



+



+#+results: tableoperationscombineintersect(upperwide,lowerwide,upperlowerwide)



+ 2  B1  B2  B3  B4  b1  b2  b3  b4  Bb1  Bb2  Bb3  Bb4 



+ 4  D1  D2  D3  D4  d1  d2  d3  d4  Dd1  Dd2  Dd3  Dd4 



+



+**** Keeping the margins



+



+ (setqdefault fillcolumn 80)



+ (columnmarker3 80)



+



+
