Better implementation of unique entry IDs.

Unique identifiers for entries can now be used more efficiently.
Internally, a hash array has replaced the alist used so far to
keep track of the files in which an ID is defined.  This makes it
quite fast to find an entry by ID.

There is now a new link type which looks like this:

id:GLOBALLY-UNIQUE-IDENTIFIER

This link points to a specific entry.  When you move the entry to
a different file, for example if you move it to an archive
file, this link will continue to work.

The file /org-id.el/ contains an API that can be used to write
code using these identifiers, including creating IDs and finding
them wherever they are.

Org has its own method to create unique identifiers, but if the
system has /uuidgen/ command installed (Mac's and Linux systems
generally do), it will be used by default.  You an also select
the method by hand, using the variable =org-id-method=.

If the ID system ever gets confused about where a certain ID is,
it initiates a global scan of all agenda files with associated
archives, all files previously known containing any IDs, and all
currently visited Org-mode files to rebuild the hash.  You can
also initiate this by hand: =M-x org-id-update-id-locations=.
Running this command will also dump into the =*Messages*= buffer
information about any duplicate IDs.  These should not exist, but
if you /copy/ an entry with its properties, duplicate IDs will
inevitably be produced.  This is unavoidable in a plain text
system that allows you to edit the text in arbitrary ways, and a
portion of care on your side is needed to keep this system clean.

The hash is stored in the file =~/.emacs.d/.org-id-locations=.
This is also a change from previous versions where the file was
=~/.org=id-locations=.  Therefore, you can remove this old file
if you have it.  I am not sure what will happen if the =.emacs.d=
directory does not exists in your setup, but in modern Emacsen, I
believe it should exist.  If you do not want to use IDs across
files, you can avoid the overhead with tracking IDs by
customizing the variable =org-id-track-globally=.  IDs can then
still be used for links inside a single file.

IDs will also be used when you create a new link to an Org-mode
buffer.  If you use =org-store-link= (normally at =C-c l=) inside
en entry in an Org-mode buffer, and ID property will be created
if it does not exist, and the stored link will be an =id:= link.
If you prefer the much less secure linking to headline text, you
can configure the variable =org-link-to-org-use-id=.  The default
setting for this variable is =create-if-interactive=, meaning
that an ID will be created when you store a link interactively,
but not if you happen to be in an Org-mode file while you create
a remember note (which usually has a link to the place where you
were when starting remember).

7 files changed, 354 insertions, 145 deletions

diff --git a/ORGWEBPAGE/Changes.org b/ORGWEBPAGE/Changes.org
index d02e4f1..1468fb3 100644
--- a/ORGWEBPAGE/Changes.org
+++ b/ORGWEBPAGE/Changes.org
@@ -10,14 +10,13 @@
 #+LINK_UP: index.html
 #+LINK_HOME: http://orgmode.org
 
-
 * Version 6.15 (in preparation)
   :PROPERTIES:
   :VISIBILITY: content
   :END:
 ** Overview
 ** Incompatible changes
-*** Old syntax for link attributes disbandoned
+*** Old syntax for link attributes abandoned
     There used to be a syntax for settint link attributes for
     HTML export by enclosing the attributes into double braces
     and adding them to the link itself, like
@@ -50,7 +49,7 @@ link.
        ID for the ~<div class="figure">~ element that encapsulates
        the image tag and the caption.  For LaTeX export, this
        string will be used as the argument of a ~\label{...}~
-       macro.  These lables will be available for internal links
+       macro.  These labels will be available for internal links
        like ~[[label][Table] ]~.
   - #+ATTR_HTML: :: Attributes for HTML export of image, to be
        added as attributes into the ~<img...>~ tag.  This string
@@ -67,6 +66,66 @@ For LaTeX export, if either a caption or a label is given, the
 element will be exported as a float, i.e. wrapped into a figure
 or table environment.
 
+*** Better implementation for entry IDs
+    
+Unique identifiers for entries can now be used more efficiently.
+Internally, a hash array has replaced the alist used so far to
+keep track of the files in which an ID is defined.  This makes it
+quite fast to find an entry by ID.
+
+There is now a new link type which looks like this:
+
+#+begin_example
+id:GLOBALLY-UNIQUE-IDENTIFIER
+#+end_example
+
+This link points to a specific entry.  When you move the entry to
+a different file, for example if you move it to an archive
+file, this link will continue to work.
+
+The file /org-id.el/ contains an API that can be used to write
+code using these identifiers, including creating IDs and finding
+them wherever they are.
+
+Org has its own method to create unique identifiers, but if the
+system has /uuidgen/ command installed (Mac's and Linux systems
+generally do), it will be used by default.  You an also select
+the method by hand, using the variable =org-id-method=.
+
+If the ID system ever gets confused about where a certain ID is,
+it initiates a global scan of all agenda files with associated
+archives, all files previously known containing any IDs, and all
+currently visited Org-mode files to rebuild the hash.  You can
+also initiate this by hand: =M-x org-id-update-id-locations=.
+Running this command will also dump into the =*Messages*= buffer
+information about any duplicate IDs.  These should not exist, but
+if you /copy/ an entry with its properties, duplicate IDs will
+inevitably be produced.  This is unavoidable in a plain text
+system that allows you to edit the text in arbitrary ways, and a
+portion of care on your side is needed to keep this system clean.
+
+The hash is stored in the file =~/.emacs.d/.org-id-locations=.
+This is also a change from previous versions where the file was
+=~/.org=id-locations=.  Therefore, you can remove this old file
+if you have it.  I am not sure what will happen if the =.emacs.d=
+directory does not exists in your setup, but in modern Emacsen, I
+believe it should exist.  If you do not want to use IDs across
+files, you can avoid the overhead with tracking IDs by
+customizing the variable =org-id-track-globally=.  IDs can then
+still be used for links inside a single file.
+
+IDs will also be used when you create a new link to an Org-mode
+buffer.  If you use =org-store-link= (normally at =C-c l=) inside
+en entry in an Org-mode buffer, and ID property will be created
+if it does not exist, and the stored link will be an =id:= link.
+If you prefer the much less secure linking to headline text, you
+can configure the variable =org-link-to-org-use-id=.  The default
+setting for this variable is =create-if-interactive=, meaning
+that an ID will be created when you store a link interactively,
+but not if you happen to be in an Org-mode file while you create
+a remember note (which usually has a link to the place where you
+were when starting remember).
+
 * Version 6.14
 ** Overview
 
@@ -275,7 +334,7 @@ or table environment.
 *** Invisible targets become now anchors in headlines.
 
     These anchors can be used to jump to a directly with an HTML
-    link, just like the =sec-xxx= ids.  For example, the
+    link, just like the =sec-xxx= IDs.  For example, the
     following will make a http link
     =//domain/path-to-my-file.html#dummy= work:
 
diff --git a/doc/org.texi b/doc/org.texi
index cb842e3..14ab4ca 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -2572,26 +2572,25 @@ insert it into an Org file, and to follow the link.
 @kindex C-c l
 @cindex storing links
 @item C-c l
-Store a link to the current location.  This is a @emph{global} command
-which can be used in any buffer to create a link.  The link will be
-stored for later insertion into an Org buffer (see below).  For
-Org files, if there is a @samp{<<target>>} at the cursor, the
-link points to the target.  Otherwise it points to the current
-headline.  For VM, Rmail, Wanderlust, MH-E, Gnus and BBDB buffers, the
-link will indicate the current article/entry.  For W3 and W3M buffers,
-the link goes to the current URL.  For IRC links, if you set the
-variable @code{org-irc-link-to-logs} to non-nil then @kbd{C-c l} will
-store a @samp{file:/} style link to the relevant point in the logs for
-the current conversation. Otherwise an @samp{irc:/} style link to the
-user/channel/server under the point will be stored.  For any other
-files, the link will point to the file, with a search string
-(@pxref{Search options}) pointing to the contents of the current line.
-If there is an active region, the selected words will form the basis
-of the search string.  If the automatically created link is not
-working correctly or accurately enough, you can write custom functions
-to select the search string and to do the search for particular file
-types - see @ref{Custom searches}.  The key binding @kbd{C-c l} is
-only a suggestion - see @ref{Installation}.
+Store a link to the current location.  This is a @emph{global} command which
+can be used in any buffer to create a link.  The link will be stored for
+later insertion into an Org buffer (see below).  For Org files, if there is a
+@samp{<<target>>} at the cursor, the link points to the target.  Otherwise it
+points to the current headline, either by text, or, if @file{org-id.el} is
+loaded, by ID property.  For VM, Rmail, Wanderlust, MH-E, Gnus and BBDB
+buffers, the link will indicate the current article/entry.  For W3 and W3M
+buffers, the link goes to the current URL.  For IRC links, if you set the
+variable @code{org-irc-link-to-logs} to non-nil then @kbd{C-c l} will store a
+@samp{file:/} style link to the relevant point in the logs for the current
+conversation. Otherwise an @samp{irc:/} style link to the user/channel/server
+under the point will be stored.  For any other files, the link will point to
+the file, with a search string (@pxref{Search options}) pointing to the
+contents of the current line.  If there is an active region, the selected
+words will form the basis of the search string.  If the automatically created
+link is not working correctly or accurately enough, you can write custom
+functions to select the search string and to do the search for particular
+file types - see @ref{Custom searches}.  The key binding @kbd{C-c l} is only
+a suggestion - see @ref{Installation}.
 @c
 @kindex C-c C-l
 @cindex link completion
diff --git a/lisp/ChangeLog b/lisp/ChangeLog
index 5b16d16..9afe342 100755
--- a/lisp/ChangeLog
+++ b/lisp/ChangeLog
@@ -32,6 +32,24 @@
 	(org-export-as-html): Close paragraph before blockquote and verse
 	tags.
 
+2008-12-06  Carsten Dominik  <carsten.dominik@gmail.com>
+
+	* org-id.el (org-id-locations-file): Wrap file name with
+	`convert-standard-filename'.
+	(org-id-files): New variable.
+	(org-id-use-hash): New option.
+	(org-id-update-id-locations): Also search in all files current
+	listed in `org-id-files'.  Convert the resulting alist to a hash
+	if the user customation says so.
+	(org-id-locations-save): Handle he case if `org-id-locations' is a
+	hash.
+	(org-id-locations-load): Convert the alist to a hash.
+	(org-id-add-location): Handle the hast case.
+	(kill-emacs-hook): Make sure id locations are saved when Emacs is
+	exited.
+	(org-id-hash-to-alist, org-id-alist-to-hash)
+	(org-id-paste-tracker): New functions.
+
 2008-12-05  Carsten Dominik  <carsten.dominik@gmail.com>
 
 	* org-agenda.el (org-agenda-goto-calendar): Remove duplicate let
diff --git a/lisp/org-clock.el b/lisp/org-clock.el
index 17fb7c2..b6fc637 100644
--- a/lisp/org-clock.el
+++ b/lisp/org-clock.el
@@ -118,7 +118,8 @@ be visited."
 	  (const :tag "Clock and history" t)
 	  (const :tag "No persistence" nil)))
 
-(defcustom org-clock-persist-file "~/.emacs.d/org-clock-save.el"
+(defcustom org-clock-persist-file (convert-standard-filename
+				   "~/.emacs.d/org-clock-save.el")
   "File to save clock data to"
   :group 'org-clock
   :type 'string)
diff --git a/lisp/org-exp.el b/lisp/org-exp.el
index 1ab6b43..2ad6743 100644
--- a/lisp/org-exp.el
+++ b/lisp/org-exp.el
@@ -1761,17 +1761,15 @@ from the buffer."
 	(todo (plist-get opts :todo-keywords))
 	(tags (plist-get opts :tags))
 	(pri  (plist-get opts :priority))
-	rpl)
+	(elts '(1 2 3 4 5))
+	rpl props)
+    (setq elts (delq nil (list 1 (if todo 2) (if pri 3) 4 (if tags 5))))
     (when (or (not todo) (not tags) (not pri))
-      ;; OK, something needs to be removed
-      (setq rpl (concat "\\1"
-			(if todo " \\2" "")
-			(if pri  " \\3" "")
-			" \\4"
-			(if tags " \\5" "")))
       (goto-char (point-min))
       (while (re-search-forward re nil t)
-	(replace-match rpl t nil)))))
+	(setq rpl (mapconcat (lambda (i) (if (match-end i) (match-string i) ""))
+			     elts " "))
+	(replace-match rpl t t)))))
 
 (defun org-export-protect-quoted-subtrees ()
   "Mark quoted subtrees with the protection property."
diff --git a/lisp/org-id.el b/lisp/org-id.el
index 04b9cb5..6c1a821 100644
--- a/lisp/org-id.el
+++ b/lisp/org-id.el
@@ -1,4 +1,4 @@
-;;; org-id.el --- Global identifier for Org-mode entries
+;;; org-id.el --- Global identifiers for Org-mode entries
 ;; Copyright (C) 2008 Free Software Foundation, Inc.
 ;;
 ;; Author: Carsten Dominik <carsten at orgmode dot org>
@@ -35,7 +35,7 @@
 ;; Org has a builtin method that uses a compact encoding of the creation
 ;; time of the ID, with microsecond accuracy.  This virtually
 ;; guarantees globally unique identifiers, even if several people are
-;; creating ID's at the same time in files that will eventually be used
+;; creating IDs at the same time in files that will eventually be used
 ;; together.  As an exernal method `uuidgen' is supported, if installed
 ;; on the system.
 ;;
@@ -78,17 +78,26 @@
   :tag "Org ID"
   :group 'org)
 
-(defcustom org-id-method 'org
-  "The method that should be used to create new ID's.
 
-An ID will consist of the prefix specified in `org-id-prefix', and a unique
-part created by the method this variable specifies.
+(defcustom org-id-method 
+  (condition-case nil
+      (if (string-match "\\`[-0-9a-fA-F]\\{36\\}\\'"
+			(org-trim (shell-command-to-string "uuidgen")))
+	  'uuidgen
+	'org)
+    (error 'org))
+  "The method that should be used to create new IDs.
+
+If `uuidgen' is available on the system, it will be used as the default method.
+if not. the methd `org' is used.
+An ID will consist of the optional prefix specified in `org-id-prefix',
+and a unique part created by the method this variable specifies.
 
 Allowed values are:
 
-org        Org's own internal method, using an encoding of the current time,
-           and the current domain of the computer.  This method will
-           honor the variable `org-id-include-domain'.
+org        Org's own internal method, using an encoding of the current time to
+           microsecond accuracy, and optionally the current domain of the
+           computer.  See the variable `org-id-include-domain'.
 
 uuidgen    Call the external command uuidgen."
   :group 'org-id
@@ -107,26 +116,54 @@ to have no space characters in them."
 	  (const :tag "No prefix")
 	  (string :tag "Prefix")))
 
-(defcustom org-id-include-domain t
+(defcustom org-id-include-domain nil
   "Non-nil means, add the domain name to new IDs.
-This ensures global uniqueness of ID's, and is also suggested by
+This ensures global uniqueness of IDs, and is also suggested by
 RFC 2445 in combination with RFC 822.  This is only relevant if
 `org-id-method' is `org'.  When uuidgen is used, the domain will never
-be added."
+be added.
+The default is to not use this because we have no really good way to get
+the true domain, and Org entries will normally not be shared with enough
+people to make this necessary."
+  :group 'org-id
+  :type 'boolean)
+
+(defcustom org-id-track-globally t
+  "Non-nil means, track IDs trhough files, so that links work globally.
+This work by maintaining a hash table for IDs and writing this table
+to disk when exiting Emacs.  Because of this, it works best if you use
+a single Emacs process, not many.
+
+When nil, IDs are not tracked.  Links to IDs will still work within
+a buffer, but not if the entry is located in another file.
+IDs can still be used if the entry with the id is in the same file as
+the link."
   :group 'org-id
   :type 'boolean)
 
 (defcustom org-id-locations-file (convert-standard-filename
-				  "~/.org-id-locations")
-  "The file for remembering the last ID number generated."
+				  "~/.emacs.d/.org-id-locations")
+  "The file for remembering in which file an ID was defined.
+This variable is only relevant when `org-id-track-globally' is set."
   :group 'org-id
   :type 'file)
 
 (defvar org-id-locations nil
-  "List of files with ID's in those files.")
+  "List of files with IDs in those files.
+Depending on `org-id-use-hash' this can also be a hash table mapping IDs
+to files.")
+
+(defvar org-id-files nil
+  "List of files that contain IDs.")
 
 (defcustom org-id-extra-files 'org-agenda-text-search-extra-files
-  "Files to be searched for ID's, besides the agenda files."
+  "Files to be searched for IDs, besides the agenda files.
+When Org reparses files to remake the list of files and IDs it is tracking,
+it will normally scan the agenda files, the archives related to agenda files,
+any files that are listed as ID containing in the current register, and
+any Org-mode files currently visited by Emacs.
+You can list additional files here.
+This variable is only relevant when `org-id-track-globally' is set."
   :group 'org-id
   :type
   '(choice
@@ -136,9 +173,9 @@ be added."
 
 (defcustom org-id-search-archives t
   "Non-nil means, search also the archive files of agenda files for entries.
-It is possible that id searches might become too slow if a user has
-used org-mode and ids for many years.  This is why it is possibl to turn this
-off."
+This is a possibility to reduce overhead, but it measn that entries moved
+to the archives can no longer be found by ID.
+This variable is only relevant when `org-id-track-globally' is set."
   :group 'org-id
   :type 'boolean)
 
@@ -210,7 +247,7 @@ It returns the ID of the entry.  If necessary, the ID is created."
 (defun org-id-goto (id)
   "Switch to the buffer containing the entry with id ID.
 Move the cursor to that entry in that buffer."
-  (interactive)
+  (interactive "sID: ")
   (let ((m (org-id-find id 'marker)))
     (unless m
       (error "Cannot find entry with ID \"%s\"" id))
@@ -335,88 +372,162 @@ and time is the usual three-integer representation of time."
 ;; Storing ID locations (files)
 
 (defun org-id-update-id-locations (&optional files)
-  "Scan relevant files for ID's.
-Store the relation between files and corresponding ID's.
+  "Scan relevant files for IDs.
+Store the relation between files and corresponding IDs.
 This will scan all agenda files, all associated archives, and all
 files currently mentioned in `org-id-locations'.
-When FILES is given, scan these files instead."
+When FILES is given, scan these files instead.
+When CHECK is given, prepare detailed iinformation about duplicate IDs."
   (interactive)
-  (let ((files
-	 (or files
-	     (append (org-agenda-files t org-id-search-archives)
-		     (if (symbolp org-id-extra-files)
-			 (symbol-value org-id-extra-files)
-		       org-id-extra-files)
-		     (mapcar 'car org-id-locations))))
-	org-agenda-new-buffers
-	file nfiles tfile ids reg found id seen)
-    (setq nfiles (length files))
-    (while (setq file (pop files))
-      (message "Finding ID locations (%d/%d files)"
-	       (- nfiles (length files)) nfiles)
-      (setq tfile (file-truename file))
-      (when (and (file-exists-p file) (not (member tfile seen)))
-	(push tfile seen)
-	(setq ids nil)
-	(with-current-buffer (org-get-agenda-file-buffer file)
-	  (save-excursion
-	    (save-restriction
-	      (widen)
-	      (goto-char (point-min))
-	      (while (re-search-forward "^[ \t]*:ID:[ \t]+\\(\\S-+\\)[ \t]*$"
-					nil t)
-		(setq id (org-match-string-no-properties 1))
-		(if (member id found)
-		    (error "Duplicate ID \"%s\"" id))
-		(push id found)
-		(push id ids))
-	      (push (cons file ids) reg))))))
-    (org-release-buffers org-agenda-new-buffers)
-    (setq org-agenda-new-buffers nil)
-    (setq org-id-locations reg)
-    (org-id-locations-save)))
+  (if (not org-id-track-globally)
+      (error "Please turn on `org-id-track-globally' if you want to track IDs.")
+    (let ((files
+	   (or files
+	       (append
+		;; Agenda files and all associated archives
+		(org-agenda-files t org-id-search-archives)
+		;; Explicit extra files
+		(if (symbolp org-id-extra-files)
+		    (symbol-value org-id-extra-files)
+		  org-id-extra-files)
+	      ;; Files associated with live org-mode buffers
+		(delq nil
+		      (mapcar (lambda (b)
+				(with-current-buffer b
+				  (and (org-mode-p) (buffer-file-name))))
+			      (buffer-list)))
+		;; All files known to have IDs
+		org-id-files)))
+	  org-agenda-new-buffers
+	  file nfiles tfile ids reg found id seen (ndup 0))
+      (setq nfiles (length files))
+      (while (setq file (pop files))
+	(message "Finding ID locations (%d/%d files): %s"
+		 (- nfiles (length files)) nfiles file)
+	(setq tfile (file-truename file))
+	(when (and (file-exists-p file) (not (member tfile seen)))
+	  (push tfile seen)
+	  (setq ids nil)
+	  (with-current-buffer (org-get-agenda-file-buffer file)
+	    (save-excursion
+	      (save-restriction
+		(widen)
+		(goto-char (point-min))
+		(while (re-search-forward "^[ \t]*:ID:[ \t]+\\(\\S-+\\)[ \t]*$"
+					  nil t)
+		  (setq id (org-match-string-no-properties 1))
+		  (if (member id found)
+		      (progn
+			(message "Duplicate ID \"%s\", also in file %s"
+				 id (car (delq
+					  nil
+					  (mapcar
+					   (lambda (x)
+					     (if (member id (cdr x)) (car x)))
+					   reg))))
+			(when (= ndup 0)
+			  (ding)
+			  (sit-for 2))
+			(setq ndup (1+ ndup)))
+		    (push id found)
+		    (push id ids)))
+		(push (cons (abbreviate-file-name file) ids) reg))))))
+      (org-release-buffers org-agenda-new-buffers)
+      (setq org-agenda-new-buffers nil)
+      (setq org-id-locations reg)
+      (setq org-id-files (mapcar 'car org-id-locations))
+      (org-id-locations-save) ;; this function can also handle the alist form
+      ;; now convert to a hash
+      (setq org-id-locations (org-id-alist-to-hash org-id-locations))
+      (if (> ndup 0)
+	  (message "WARNING: %d duplicate IDs found, check *Messages* buffer" ndup)
+	(message "%d unique files scanned for IDs" (length org-id-files)))
+      org-id-locations)))
 
 (defun org-id-locations-save ()
   "Save `org-id-locations' in `org-id-locations-file'."
-  (with-temp-file org-id-locations-file
-    (print org-id-locations (current-buffer))))
+  (when org-id-track-globally
+    (let ((out (if (hash-table-p org-id-locations)
+		   (org-id-hash-to-alist org-id-locations)
+		 org-id-locations)))
+      (with-temp-file org-id-locations-file
+	(print out (current-buffer))))))
 
 (defun org-id-locations-load ()
   "Read the data from `org-id-locations-file'."
   (setq org-id-locations nil)
-  (with-temp-buffer
-    (condition-case nil
-        (progn
-          (insert-file-contents-literally org-id-locations-file)
-          (goto-char (point-min))
-          (setq org-id-locations (read (current-buffer))))
-      (error
-       (message "Could not read org-id-values from %s. Setting it to nil."
-                org-id-locations-file)))))
+  (when org-id-track-globally
+    (with-temp-buffer
+      (condition-case nil
+	  (progn
+	    (insert-file-contents-literally org-id-locations-file)
+	    (goto-char (point-min))
+	    (setq org-id-locations (read (current-buffer))))
+	(error
+	 (message "Could not read org-id-values from %s. Setting it to nil."
+		  org-id-locations-file))))
+    (setq org-id-files (mapcar 'car org-id-locations))
+    (setq org-id-locations (org-id-alist-to-hash org-id-locations))))
 
 (defun org-id-add-location (id file)
   "Add the ID with location FILE to the database of ID loations."
-  (when (and id file) ; don't error when called from a buffer with no file
+  ;; Only if global tracking is on, and when the buffer has a file
+  (when (and org-id-track-globally id file) 
     (unless org-id-locations (org-id-locations-load))
-    (catch 'exit
-      (let ((locs org-id-locations) list)
-	(while (setq list (pop locs))
-	  (when (equal (file-truename file) (file-truename (car list)))
-	    (setcdr list (cons id (cdr list)))
-	    (throw 'exit t))))
-      (push (list file id) org-id-locations))
-    (org-id-locations-save)))
+    (puthash id (abbreviate-file-name file) org-id-locations)
+    (add-to-list 'org-id-files (abbreviate-file-name file))))
+
+(add-hook 'kill-emacs-hook 'org-id-locations-save)
+
+(defun org-id-hash-to-alist (hash)
+  "Turn an org-id hash into an alist, so that it can be written to a file."
+  (let (res x)
+    (maphash
+     (lambda (k v)
+       (if (setq x (member v res))
+	   (push k (cdr x))
+	 (push (list v k) res)))
+     hash)
+    res))
+
+(defun org-id-alist-to-hash (list)
+  "Turn an org-id location list into a hash table."
+  (let ((res (make-hash-table
+	      :test 'equal
+	      :size (apply '+ (mapcar 'length list))))
+	f i)
+    (mapc
+     (lambda (x)
+       (setq f (car x))
+       (mapc (lambda (i) (puthash i f res)) (cdr x)))
+     list)
+    res))
+
+(defun org-id-paste-tracker (txt &optional buffer-or-file)
+  "Update any IDs in TXT and assign BUFFER-OR-FILE to them."
+  (when org-id-track-globally
+    (save-match-data
+      (setq buffer-or-file (or buffer-or-file (current-buffer)))
+      (when (bufferp buffer-or-file)
+	(setq buffer-or-file (or (buffer-base-buffer buffer-or-file)
+				 buffer-or-file))
+	(setq buffer-or-file (buffer-file-name buffer-or-file)))
+      (when buffer-or-file
+	(let ((fname (abbreviate-file-name buffer-or-file))
+	      (s 0))
+	  (while (string-match "^[ \t]*:ID:[ \t]+\\([^ \t\n\r]+\\)" txt s)
+	    (setq s (match-end 0))
+	    (org-id-add-location (match-string 1 txt) fname)))))))
 
 ;; Finding entries with specified id
 
 (defun org-id-find-id-file (id)
   "Query the id database for the file in which this ID is located."
   (unless org-id-locations (org-id-locations-load))
-  (catch 'found
-    (mapc (lambda (x) (if (member id (cdr x))
-			  (throw 'found (car x))))
-	  org-id-locations)
-    nil))
+  (or (gethash id org-id-locations)
+      ;; ball back on current buffer
+      (buffer-file-name (or (buffer-base-buffer (current-buffer))
+			    (current-buffer)))))
 
 (defun org-id-find-id-in-file (id file &optional markerp)
   "Return the position of the entry ID in FILE.
@@ -435,7 +546,10 @@ optional argument MARKERP, return the position as a new marker."
 		(move-marker (make-marker) pos buf)
 	      (cons file pos))))))))
 
-(org-add-link-type "id" 'org-id-open)
+;; id link type
+
+;; Calling the following function is hard-coded into `org-store-link',
+;; so we do have to add it to `org-store-link-functions'.
 
 (defun org-id-store-link ()
   "Store a link to the current entry, using it's ID."
@@ -452,8 +566,16 @@ optional argument MARKERP, return the position as a new marker."
 (defun org-id-open (id)
   "Go to the entry with id ID."
   (org-mark-ring-push)
-  (switch-to-buffer-other-window (current-buffer))
-  (org-id-goto id))
+  (let ((m (org-id-find id 'marker)))
+    (unless m
+      (error "Cannot find entry with ID \"%s\"" id))
+    (if (not (equal (current-buffer) (marker-buffer m)))
+	(switch-to-buffer-other-window (marker-buffer m)))
+    (goto-char m)
+    (move-marker m nil)
+    (org-show-context)))
+
+(org-add-link-type "id" 'org-id-open)
 
 (provide 'org-id)
 
@@ -461,3 +583,4 @@ optional argument MARKERP, return the position as a new marker."
 
 ;; arch-tag: e5abaca4-e16f-4b25-832a-540cfb63a712
 
+
diff --git a/lisp/org.el b/lisp/org.el
index 984864c..e4bb108 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -162,7 +162,7 @@ to add the symbol `xyz', and the package must have a call to
 	(const :tag "   bbdb:              Links to BBDB entries" org-bbdb)
 	(const :tag "   bibtex:            Links to BibTeX entries" org-bibtex)
 	(const :tag "   gnus:              Links to GNUS folders/messages" org-gnus)
-	(const :tag "   id:                Global id's for identifying entries" org-id)
+	(const :tag "   id:                Global IDs for identifying entries" org-id)
 	(const :tag "   info:              Links to Info nodes" org-info)
 	(const :tag "   jsinfo:            Set up Sebastian Rose's JavaScript org-info.js" org-jsinfo)
 	(const :tag "   irc:               Links to IRC/ERC chat sessions" org-irc)
@@ -957,7 +957,11 @@ It should match if the message is from the user him/herself."
   :type 'regexp)
 
 (defcustom org-link-to-org-use-id 'create-if-interactive
-  "Non-nil means, storing a link to an Org file will use entry ID's.
+  "Non-nil means, storing a link to an Org file will use entry IDs.
+
+Note that before this variable is even considered, org-id must be loaded,
+to please customize `org-modules' and turn it on.
+
 The variable can have the following values:
 
 t     Create an ID if needed to make a link to the current entry.
@@ -967,7 +971,7 @@ create-if-interactive
       command), do create an ID to support the link.  But when doing the
       job for remember, only use the ID if it already exists.  The
       purpose of this setting is to avoid proliferation of unwanted
-      ID's, just because you happen to be in an Org file when you
+      IDs, just because you happen to be in an Org file when you
       call `org-remember' that automatically and preemptively
       creates a link.  If you do want to get an ID link in a remember
       template to an entry not having an ID, create it first by
@@ -5299,6 +5303,7 @@ the inserted text when done."
     (beginning-of-line 1)
     (unless for-yank (org-back-over-empty-lines))
     (setq beg (point))
+    (and (fboundp 'org-id-paste-tracker) (org-id-paste-tracker txt))
     (insert-before-markers txt)
     (unless (string-match "\n\\'" txt) (insert "\n"))
     (setq newend (point))
@@ -6172,7 +6177,6 @@ type.  For a simple example of an export function, see `org-bbdb.el'."
       (setcdr (assoc type org-link-protocols) (list follow export))
     (push (list type follow export) org-link-protocols)))
 
-
 ;;;###autoload
 (defun org-store-link (arg)
   "\\<org-mode-map>Store an org-link to the current location.
@@ -6234,11 +6238,20 @@ For file links, arg negates `org-context-in-file-links'."
 
      ((and buffer-file-name (org-mode-p))
       (cond
-       ((or (eq org-link-to-org-use-id t)
-	    (and (eq org-link-to-org-use-id 'create-if-interactive)
-		 (interactive-p))
-	    (and org-link-to-org-use-id
-		 (condition-case nil (org-entry-get nil "ID") (error nil))))
+       ((org-in-regexp "<<\\(.*?\\)>>")
+	(setq cpltxt
+	      (concat "file:"
+		      (abbreviate-file-name buffer-file-name)
+		      "::" (match-string 1))
+	      link (org-make-link cpltxt)))
+       ((and (featurep 'org-id)
+	     (or (eq org-link-to-org-use-id t)
+		 (and (eq org-link-to-org-use-id 'create-if-interactive)
+		      (interactive-p))
+		 (and org-link-to-org-use-id
+		      (condition-case nil
+			  (org-entry-get nil "ID")
+			(error nil)))))
 	;; We can make a link using the ID.
 	(setq link (condition-case nil
 		       (org-id-store-link)
@@ -6252,24 +6265,21 @@ For file links, arg negates `org-context-in-file-links'."
 			     (abbreviate-file-name buffer-file-name)))
 	;; Add a context search string
 	(when (org-xor org-context-in-file-links arg)
-	  ;; Check if we are on a target
-	  (if (org-in-regexp "<<\\(.*?\\)>>")
-	      (setq cpltxt (concat cpltxt "::" (match-string 1)))
-	    (setq txt (cond
-		       ((org-on-heading-p) nil)
-		       ((org-region-active-p)
-			(buffer-substring (region-beginning) (region-end)))
-		       (t nil)))
-	    (when (or (null txt) (string-match "\\S-" txt))
-	      (setq cpltxt
-		    (concat cpltxt "::"
-			    (condition-case nil
-				(org-make-org-heading-search-string txt)
-			      (error "")))
-		    desc "NONE")))))
-       (if (string-match "::\\'" cpltxt)
-	   (setq cpltxt (substring cpltxt 0 -2)))
-       (setq link (org-make-link cpltxt))))
+	  (setq txt (cond
+		     ((org-on-heading-p) nil)
+		     ((org-region-active-p)
+		      (buffer-substring (region-beginning) (region-end)))
+		     (t nil)))
+	  (when (or (null txt) (string-match "\\S-" txt))
+	    (setq cpltxt
+		  (concat cpltxt "::"
+			  (condition-case nil
+			      (org-make-org-heading-search-string txt)
+			    (error "")))
+		  desc "NONE")))
+	(if (string-match "::\\'" cpltxt)
+	    (setq cpltxt (substring cpltxt 0 -2)))
+	(setq link (org-make-link cpltxt)))))
 
      ((buffer-file-name (buffer-base-buffer))
       ;; Just link to this file here.
@@ -10606,6 +10616,7 @@ completion."
 IDENT can be a string, a symbol or a number, this function will search for
 the string representation of it.
 Return the position where this entry starts, or nil if there is no such entry."
+  (interactive "sID: ")
   (let ((id (cond
 	     ((stringp ident) ident)
 	     ((symbol-name ident) (symbol-name ident))