|
ABSTRACT
Data sets in large applications are often too massive to fit completely inside the computer's internal memory. The resulting input/output communication (or I/O) between fast internal memory and slower external memory (such as disks) can be a major performance bottleneck. In this manuscript, we survey the state of the art in the design and analysis of algorithms and data structures for external memory (or EM for short), where the goal is to exploit locality and parallelism in order to reduce the I/O costs. We consider a variety of EM paradigms for solving batched and online problems efficiently in external memory. For the batched problem of sorting and related problems like permuting and fast Fourier transform, the key paradigms include distribution and merging. The paradigm of disk striping offers an elegant way to use multiple disks in parallel. For sorting, however, disk striping can be nonoptimal with respect to I/O, so to gain further improvements we discuss distribution and merging techniques for using the disks independently. We also consider useful techniques for batched EM problems involving matrices, geometric data, and graphs. In the online domain, canonical EM applications include dictionary lookup and range searching. The two important classes of indexed data structures are based upon extendible hashing and B-trees. The paradigms of filtering and bootstrapping provide convenient means in online data structures to make effective use of the data accessed from disk. We also re-examine some of the above EM problems in slightly different settings, such as when the data items are moving, when the data items are variable-length such as character strings, when the data structure is compressed to save space, or when the allocated amount of internal memory can change dynamically. Programming tools and environments are available for simplifying the EM programming task. We report on some experiments in the domain of spatial databases using the TPIE system (Transparent Parallel I/O programming Environment). The newly developed EM algorithms and data structures that incorporate the paradigms we discuss are significantly faster than other methods used in practice.
REFERENCES
Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.
| |
1
|
D. J. Abel, "A B+-tree structure for large quadtrees," Computer Vision, Graphics, and Image Processing, vol. 27, pp. 19-31, July 1984.
|
| |
2
|
J. Abello, A. L. Buchsbaum, and J. Westbrook, "A functional approach to external graph algorithms," Algorithmica, vol. 32, no. 3, pp. 437-458, 2002.
|
| |
3
|
|
 |
4
|
|
| |
5
|
|
| |
6
|
Pankaj K. Agarwal , Lars Arge , Gerth Stølting Brodal , Jeffrey S. Vitter, I/O-efficient dynamic point location in monotone planar subdivisions, Proceedings of the tenth annual ACM-SIAM symposium on Discrete algorithms, p.11-20, January 17-19, 1999, Baltimore, Maryland, United States
|
| |
7
|
P. K. Agarwal, L. Arge, and A. Danner, "From LIDAR to GRID DEM: A scalable approach," in Proceedings of the International Symposium on Spatial Data Handling, 2006.
|
| |
8
|
|
| |
9
|
|
| |
10
|
Pankaj K. Agarwal , Lars Arge , T. M. Murali , Kasturi R. Varadarajan , Jeffrey Scott Vitter, I/O-efficient algorithms for contour-line extraction and planar graph blocking, Proceedings of the ninth annual ACM-SIAM symposium on Discrete algorithms, p.117-126, January 25-27, 1998, San Francisco, California, United States
|
| |
11
|
Pankaj K. Agarwal , Lars Arge , Octavian Procopiuc , Jeffrey Scott Vitter, A Framework for Index Bulk Loading and Dynamization, Proceedings of the 28th International Colloquium on Automata, Languages and Programming,, p.115-127, July 08-12, 2001
|
| |
12
|
|
| |
13
|
P. K. Agarwal, L. Arge, J. Yang, and K. Yi, "I/O-efficient structures for orthogonal range-max and stabbing-max queries," in Proceedings of the European Symposium on Algorithms, pp. 7-18, Springer-Verlag, 2003.
|
| |
14
|
P. K. Agarwal, L. Arge, and K. Yi, "I/O-efficient construction of constrained delaunay triangulations," in Proceedings of the European Symposium on Algorithms , pp. 355-366, Springer-Verlag, 2005.
|
| |
15
|
|
 |
16
|
|
| |
17
|
P. K. Agarwal, M. de Berg, J. Gudmundsson, M. Hammar, and H. J. Haverkort, "Box-trees and R-trees with near-optimal query time," Discrete and Computational Geometry, vol. 28, no. 3, pp. 291-312, 2002.
|
| |
18
|
P. K. Agarwal and J. Erickson, "Geometric range searching and its relatives," in Advances in Discrete and Computational Geometry, (B. Chazelle, J. E. Goodman, and R. Pollack, eds.), pp. 1-56, Providence, Rhode Island: American Mathematical Society Press, 1999.
|
| |
19
|
|
 |
20
|
A. Aggarwal , B. Alpern , A. Chandra , M. Snir, A model for hierarchical memory, Proceedings of the nineteenth annual ACM symposium on Theory of computing, p.305-314, January 1987, New York, New York, United States
[doi> 10.1145/28395.28428]
|
| |
21
|
A. Aggarwal, A. Chandra, and M. Snir, "Hierarchical memory with block transfer," in Proceedings of the IEEE Symposium on Foundations of Computer Science, pp. 204-216, Los Angeles, 1987.
|
| |
22
|
|
 |
23
|
|
| |
24
|
|
| |
25
|
|
| |
26
|
D. Ajwani, I. Malinger, U. Meyer, and S. Toledo, "Characterizing the performance of flash memory storage devices and its impact on algorithm design," in Proceedings of the International Workshop on Experimental Algorithmics, (Provincetown, Mass.), pp. 208-219, Springer-Verlag, 2008.
|
| |
27
|
D. Ajwani, U. Meyer, and V. Osipov, "Improved external memory BFS implementation," in Proceedings of the Workshop on Algorithm Engineering and Experiments, (New Orleans), pp. 3-12, January 2007.
|
| |
28
|
|
| |
29
|
B. Alpern, L. Carter, E. Feig, and T. Selker, "The uniform memory hierarchy model of computation," Algorithmica, vol. 12, no. 2-3, pp. 72-109, 1994.
|
| |
30
|
|
| |
31
|
|
| |
32
|
L. Arge, "The buffer tree: A technique for designing batched external data structures," Algorithmica, vol. 37, no. 1, pp. 1-24, 2003.
|
| |
33
|
L. Arge, G. Brodal, and R. Fagerberg, "Cache-oblivious data structures," in Handbook on Data Structures and Applications, (D. Mehta and S. Sahni, eds.), CRC Press, 2005.
|
| |
34
|
|
| |
35
|
Lars Arge , Jeffrey S. Chase , Patrick Halpin , Laura Toma , Jeffrey S. Vitter , Dean Urban , Rajiv Wickremesinghe, Efficient Flow Computation on Massive Grid Terrain Datasets, Geoinformatica, v.7 n.4, p.283-313, December 2003
[doi> 10.1023/A:1025526421410]
|
| |
36
|
L. Arge, A. Danner, H. J. Haverkort, and N. Zeh, "I/O-efficient hierarchical watershed decomposition of grid terrains models," in Proceedings of the International Symposium on Spatial Data Handling, 2006.
|
| |
37
|
L. Arge, A. Danner, and S.-H. Teh, "I/O-efficient point location using persistent B-trees," in Workshop on Algorithm Engineering and Experimentation, 2003.
|
 |
38
|
|
 |
39
|
Lars Arge , Paolo Ferragina , Roberto Grossi , Jeffrey Scott Vitter, On sorting strings in external memory (extended abstract), Proceedings of the twenty-ninth annual ACM symposium on Theory of computing, p.540-548, May 04-06, 1997, El Paso, Texas, United States
[doi> 10.1145/258533.258647]
|
| |
40
|
L. Arge, M. T. Goodrich, M. Nelson, and N. Sitchinava, "Fundamental parallel algorithms for private-cache chip multiprocessors," in Proceedings of the ACM Symposium on Parallel Algorithms and Architectures, Munich: ACM Press, June 2008.
|
| |
41
|
L. Arge, K. H. Hinrichs, J. Vahrenhold, and J. S. Vitter, "Efficient bulk operations on dynamic R-trees," Algorithmica, vol. 33, pp. 104-128, January 2002.
|
| |
42
|
|
| |
43
|
L. Arge, U. Meyer, and L. Toma, "External memory algorithms for diameter and all-pairs shortest-paths on sparse graphs," in Proceedings of the International Colloquium on Automata, Languages, and Programming, pp. 146-157, Springer-Verlag, 2004.
|
| |
44
|
L. Arge, U. Meyer, L. Toma, and N. Zeh, "On external-memory planar depth first search," Journal of Graph Algorithms and Applications, vol. 7, no. 2, pp. 105-129, 2003.
|
| |
45
|
|
| |
46
|
Lars Arge , Octavian Procopiuc , Sridhar Ramaswamy , Torsten Suel , Jan Vahrenhold , Jeffrey Scott Vitter, A Unified Approach for Indexed and Non-Indexed Spatial Joins, Proceedings of the 7th International Conference on Extending Database Technology: Advances in Database Technology, p.413-429, March 27-31, 2000
|
| |
47
|
|
| |
48
|
Lars Arge , Octavian Procopiuc , Sridhar Ramaswamy , Torsten Suel , Jeffrey Scott Vitter, Theory and practice of I/O-efficient algorithms for multidimensional batched searching problems, Proceedings of the ninth annual ACM-SIAM symposium on Discrete algorithms, p.685-694, January 25-27, 1998, San Francisco, California, United States
|
| |
49
|
|
 |
50
|
Lars Arge , Vasilis Samoladas , Jeffrey Scott Vitter, On two-dimensional indexability and optimal range search indexing, Proceedings of the eighteenth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, p.346-357, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
[doi> 10.1145/303976.304010]
|
| |
51
|
L. Arge, V. Samoladas, and K. Yi, "Optimal external memory planar point enclosure," in Proceedings of the European Symposium on Algorithms, pp. 40-52, Springer-Verlag, 2004.
|
| |
52
|
L. Arge and L. Toma, "Simplified external memory algorithms for planar DAGs," in Proceedings of the Scandinavian Workshop on Algorithmic Theory, pp. 493-503, 2004.
|
| |
53
|
L. Arge and L. Toma, "External data structures for shortest path queries on planar digraphs," in Proceedings of the International Symposium on Algorithms and Computation, pp. 328-338, Springer-Verlag, 2005.
|
| |
54
|
L. Arge, L. Toma, and J. S. Vitter, "I/O-efficient algorithms for problems on grid-based terrains," in Workshop on Algorithm Engineering and Experimentation , San Francisco: Springer-Verlag, January 2000.
|
 |
55
|
|
| |
56
|
|
| |
57
|
|
| |
58
|
|
| |
59
|
|
 |
60
|
|
| |
61
|
D. Arroyuelo and G. Navarro, "A Lempel-Ziv text index on secondary storage," in Proceedings of the Symposium on Combinatorial Pattern Matching, pp. 83-94, Springer-Verlag, 2007.
|
| |
62
|
|
| |
63
|
|
| |
64
|
|
| |
65
|
|
| |
66
|
|
| |
67
|
|
| |
68
|
|
| |
69
|
|
 |
70
|
Rakesh Barve , Elizabeth Shriver , Phillip B. Gibbons , Bruce K. Hillyer , Yossi Matias , Jeffrey Scott Vitter, Modeling and optimizing I/O throughput of multiple disks on a bus, Proceedings of the 1999 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.83-92, May 01-04, 1999, Atlanta, Georgia, United States
|
| |
71
|
|
| |
72
|
R. D. Barve and J. S. Vitter, "A simple and efficient parallel disk mergesort," ACM Trans. Computer Systems, vol. 35, pp. 189-215, March/April 2002.
|
| |
73
|
|
| |
74
|
R. Bayer and E. McCreight, "Organization of large ordered indexes," Acta Informatica, vol. 1, pp. 173-189, 1972.
|
 |
75
|
|
| |
76
|
|
 |
77
|
Norbert Beckmann , Hans-Peter Kriegel , Ralf Schneider , Bernhard Seeger, The R*-tree: an efficient and robust access method for points and rectangles, Proceedings of the 1990 ACM SIGMOD international conference on Management of data, p.322-331, May 23-26, 1990, Atlantic City, New Jersey, United States
|
| |
78
|
A. L. Belady, "A study of replacement algorithms for virtual storage computers," IBM Systems Journal, vol. 5, pp. 78-101, 1966.
|
| |
79
|
|
 |
80
|
|
 |
81
|
Michael A. Bender , Jeremy T. Fineman , Seth Gilbert , Bradley C. Kuszmaul, Concurrent cache-oblivious b-trees, Proceedings of the seventeenth annual ACM symposium on Parallelism in algorithms and architectures, July 18-20, 2005, Las Vegas, Nevada, USA
[doi> 10.1145/1073970.1074009]
|
 |
82
|
|
| |
83
|
J. L. Bentley and J. B. Saxe, "Decomposable searching problems I: Static-to-dynamic transformations," Journal of Algorithms, vol. 1, pp. 301-358, December 1980.
|
| |
84
|
|
 |
85
|
Mette Berger , Esben Rune Hansen , Rasmus Pagh , Mihai Pǎtraşcu , Milan Ružić , Peter Tiedemann, Deterministic load balancing and dictionaries in the parallel disk model, Proceedings of the eighteenth annual ACM symposium on Parallelism in algorithms and architectures, July 30-August 02, 2006, Cambridge, Massachusetts, USA
[doi> 10.1145/1148109.1148160]
|
| |
86
|
R. Bhatia, R. K. Sinha, and C.-M. Chen, "A hierarchical technique for constructing efficient declustering schemes for range queries," The Computer Journal, vol. 46, no. 4, pp. 358-377, 2003.
|
| |
87
|
N. Blum and K. Mehlhorn, "On the average number of rebalancing operations in weight-balanced trees," Theoretical Computer Science, vol. 11, pp. 303-320, July 1980.
|
 |
88
|
|
| |
89
|
C. Breimann and J. Vahrenhold, "External memory computational geometry revisited," in Algorithms for Memory Hierarchies, pp. 110-148, 2002.
|
 |
90
|
|
| |
91
|
|
| |
92
|
|
| |
93
|
Adam L. Buchsbaum , Michael Goldwasser , Suresh Venkatasubramanian , Jeffery R. Westbrook, On external memory graph traversal, Proceedings of the eleventh annual ACM-SIAM symposium on Discrete algorithms, p.859-860, January 09-11, 2000, San Francisco, California, United States
|
| |
94
|
|
 |
95
|
Pei Cao , Edward W. Felten , Anna R. Karlin , Kai Li, Implementation and performance of integrated application-controlled file caching, prefetching, and disk scheduling, ACM Transactions on Computer Systems (TOCS), v.14 n.4, p.311-343, Nov. 1996
[doi> 10.1145/235543.235544]
|
| |
96
|
|
| |
97
|
G. Chaudhry and T. H. Cormen, "Oblivious vs. distribution-based sorting: An experimental evaluation," in Proceedings of the European Symposium on Algorithms, pp. 317-328, Springer-Verlag, 2005.
|
| |
98
|
|
 |
99
|
|
| |
100
|
B. Chazelle and H. Edelsbrunner, "Linear space data structures for two types of range search," Discrete and Computational Geometry, vol. 2, pp. 113-126, 1987.
|
 |
101
|
Peter M. Chen , Edward K. Lee , Garth A. Gibson , Randy H. Katz , David A. Patterson, RAID: high-performance, reliable secondary storage, ACM Computing Surveys (CSUR), v.26 n.2, p.145-185, June 1994
[doi> 10.1145/176979.176981]
|
| |
102
|
Reynold Cheng , Yuni Xia , Sunil Prabhakar , Rahul Shah , Jeffrey Scott Vitter, Efficient indexing methods for probabilistic threshold queries over uncertain data, Proceedings of the Thirtieth international conference on Very large data bases, p.876-887, August 31-September 03, 2004, Toronto, Canada
|
 |
103
|
Reynold Cheng , Sarvjeet Singh , Sunil Prabhakar , Rahul Shah , Jeffrey Scott Vitter , Yuni Xia, Efficient join processing over uncertain data, Proceedings of the 15th ACM international conference on Information and knowledge management, November 06-11, 2006, Arlington, Virginia, USA
[doi> 10.1145/1183614.1183719]
|
| |
104
|
|
| |
105
|
Yi-Jen Chiang , Michael T. Goodrich , Edward F. Grove , Roberto Tamassia , Darren Erik Vengroff , Jeffrey Scott Vitter, External-memory graph algorithms, Proceedings of the sixth annual ACM-SIAM symposium on Discrete algorithms, p.139-149, January 22-24, 1995, San Francisco, California, United States
|
| |
106
|
|
| |
107
|
Y.-F. Chien, W.-K. Hon, R. Shah, and J. S. Vitter, "Geometric Burrows-Wheeler transform: Linking range searching and text indexing," in Proceedings of the Data Compression Conference, Snowbird, Utah: IEEE Computer Society Press, March 2008.
|
| |
108
|
|
| |
109
|
|
| |
110
|
|
| |
111
|
|
| |
112
|
F. Claude and G. Navarro, "A fast and compact Web graph representation," in Proceedings of the International Symposium on String Processing and Information Retrieval, pp. 105-116, Springer-Verlag, 2007.
|
| |
113
|
|
 |
114
|
|
| |
115
|
P. Corbett, D. Feitelson, S. Fineberg, Y. Hsu, B. Nitzberg, J.-P. Prost, M. Snir, B. Traversat, and P. Wong, "Overview of the MPI-IO parallel I/O interface," in Input/Output in Parallel and Distributed Computer Systems, (R. Jain, J. Werth, and J. C. Browne, eds.), ch. 5, pp. 127-146, Kluwer Academic Publishers, 1996.
|
 |
116
|
|
| |
117
|
T. H. Cormen and E. R. Davidson, "FG: A framework generator for hiding latency in parallel programs running on clusters," in Proceedings of the 17th International Conference on Parallel and Distributed Computing Systems , pp. 137-144, Sep. 2004.
|
| |
118
|
|
| |
119
|
|
| |
120
|
|
| |
121
|
A. Crauser and P. Ferragina, "A theoretical and experimental study on the construction of suffix arrays in external memory," Algorithmica, vol. 32, no. 1, pp. 1-35, 2002.
|
| |
122
|
A. Crauser , P. Ferragina , K. Mehlhorn , U. Meyer , E. A. Ramos, I/O-optimal computation of segment intersections, External memory algorithms, American Mathematical Society, Boston, MA, 1999
|
| |
123
|
A. Crauser, P. Ferragina, K. Mehlhorn, U. Meyer, and E. A. Ramos, "Randomized external-memory algorithms for line segment intersection and other geometric problems," International Journal of Computational Geometry and Applications, vol. 11, no. 3, pp. 305-337, 2001.
|
| |
124
|
|
 |
125
|
Kenneth M. Curewitz , P. Krishnan , Jeffrey Scott Vitter, Practical prefetching via data compression, Proceedings of the 1993 ACM SIGMOD international conference on Management of data, p.257-266, May 25-28, 1993, Washington, D.C., United States
|
| |
126
|
|
| |
127
|
E. R. Davidson, FG: Improving Parallel Programs and Parallel Programming Since 2003. PhD thesis, Dartmouth College Department of Computer Science, Aug. 2006.
|
| |
128
|
|
| |
129
|
|
| |
130
|
|
| |
131
|
F. K. H. A. Dehne, W. Dittrich, and D. A. Hutchinson, "Efficient External Memory Algorithms by Simulating Coarse-Grained Parallel Algorithms," Algorithmica, vol. 36, no. 2, pp. 97-122, 2003.
|
| |
132
|
F. K. H. A. Dehne, W. Dittrich, D. A. Hutchinson, and A. Maheshwari, "Bulk synchronous parallel algorithms for the external memory model," Theory of Computing Systems, vol. 35, no. 6, pp. 567-597, 2002.
|
| |
133
|
R. Dementiev, Algorithm Engineering for Large Data Sets. PhD thesis, Saarland University, 2006.
|
| |
134
|
R. Dementiev, J. Kärkkäinen, J. Mehnert, and P. Sanders, "Better external memory suffix array construction," ACM Journal of Experimental Algorithmics , in press.
|
| |
135
|
|
 |
136
|
|
| |
137
|
R. Dementiev, P. Sanders, D. Schultes, and J. Sibeyn, "Engineering an external memory minimum spanning tree algorithm," in Proceedings of IFIP International Conference on Theoretical Computer Science, Toulouse: Kluwer Academic Publishers, 2004.
|
| |
138
|
H. B. Demuth, Electronic data sorting. PhD thesis, Stanford University, 1956.
|
| |
139
|
|
| |
140
|
|
| |
141
|
W. Dittrich, D. A. Hutchinson, and A. Maheshwari, "Blocking in parallel multisearch problems," Theory of Computing Systems, vol. 34, no. 2, pp. 145-189, 2001.
|
| |
142
|
|
| |
143
|
|
| |
144
|
H. Edelsbrunner, "A new approach to rectangle intersections, Part I," International Journal of Computer Mathematics, vol. 13, pp. 209-219, 1983.
|
| |
145
|
H. Edelsbrunner, "A New approach to rectangle intersections, Part II," International Journal of Computer Mathematics, vol. 13, pp. 221-229, 1983.
|
 |
146
|
Mohamed Y. Eltabakh , Wing-Kai Hon , Rahul Shah , Walid G. Aref , Jeffrey S. Vitter, The SBC-tree: an index for run-length compressed sequences, Proceedings of the 11th international conference on Extending database technology: Advances in database technology, March 25-29, 2008, Nantes, France
[doi> 10.1145/1353343.1353407]
|
 |
147
|
|
| |
148
|
"NASA's Earth Observing System (EOS) web page, NASA Goddard Space Flight Center," http://eospso.gsfc.nasa.gov/.
|
 |
149
|
|
| |
150
|
|
| |
151
|
|
| |
152
|
R. Fagerberg, A. Pagh, and R. Pagh, "External string sorting: Faster and cache oblivious," in Proceedings of the Symposium on Theoretical Aspects of Computer Science, pp. 68-79, Springer-Verlag, 2006.
|
 |
153
|
|
 |
154
|
|
| |
155
|
|
| |
156
|
W. Feller, An Introduction to Probability Theory and its Applications. Vol. 1, New York: John Wiley & Sons, 3rd ed., 1968.
|
| |
157
|
|
 |
158
|
|
| |
159
|
P. Ferragina, R. Grossi, A. Gupta, R. Shah, and J. S. Vitter, "On searching compressed string collections cache-obliviously," in Proceedings of the ACM Conference on Principles of Database Systems, Vancouver: ACM Press, June 2008.
|
| |
160
|
|
| |
161
|
|
 |
162
|
|
 |
163
|
|
| |
164
|
P. Flajolet, "On the performance evaluation of extendible hashing and trie searching," Acta Informatica, vol. 20, no. 4, pp. 345-369, 1983.
|
| |
165
|
R. W. Floyd, "Permuting information in idealized two-level storage," in Complexity of Computer Computations, (R. Miller and J. Thatcher, eds.), pp. 105-109, Plenum, 1972.
|
| |
166
|
|
| |
167
|
|
| |
168
|
|
 |
169
|
|
 |
170
|
|
| |
171
|
G. R. Ganger, "Generating representative synthetic workloads: An unsolved problem," in Proceedings of the Computer Measurement Group Conference, pp. 1263-1269, December 1995.
|
| |
172
|
M. Gardner, ch. 7, Magic Show. New York: Knopf, 1977.
|
 |
173
|
|
| |
174
|
|
 |
175
|
|
| |
176
|
|
| |
177
|
|
| |
178
|
R. González and G. Navarro, "A compressed text index on secondary memory," in Proceedings of the International Workshop on Combinatorial Algorithms , (Newcastle, Australia), pp. 80-91, College Publications, 2007.
|
| |
179
|
M. T. Goodrich, J.-J. Tsay, D. E. Vengroff, and J. S. Vitter, "External-memory computational geometry," in Proceedings of the IEEE Symposium on Foundations of Computer Science, pp. 714-723, Palo Alto: IEEE Computer Society Press, November 1993.
|
| |
180
|
"Google Earth online database of satellite images," Available on the World-Wide Web at http://earth.google.com/.
|
| |
181
|
|
| |
182
|
|
| |
183
|
|
 |
184
|
John Linwood Griffin , Steven W. Schlosser , Gregory R. Ganger , David F. Nagle, Modeling and performance of MEMS-based storage devices, Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.56-65, June 18-21, 2000, Santa Clara, California, United States
|
| |
185
|
|
| |
186
|
|
| |
187
|
|
| |
188
|
S. K. S. Gupta, Z. Li, and J. H. Reif, "Generating efficient programs for two-level memories from tensor-products," in Proceedings of the IASTED/ISMM International Conference on Parallel and Distributed Computing and Systems, pp. 510-513, October 1995.
|
| |
189
|
|
 |
190
|
|
| |
191
|
H. J. Haverkort and L. Toma, "I/O-efficient algorithms on near-planar graphs," in Proceedings of the Latin American Theoretical Informatics Symposium , pp. 580-591, 2006.
|
 |
192
|
|
 |
193
|
Joseph M. Hellerstein , Elias Koutsoupias , Christos H. Papadimitriou, On the analysis of indexing schemes, Proceedings of the sixteenth ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systems, p.249-256, May 11-15, 1997, Tucson, Arizona, United States
[doi> 10.1145/263661.263688]
|
| |
194
|
L. Hellerstein, G. Gibson, R. M. Karp, R. H. Katz, and D. A. Patterson, "Coding techniques for handling failures in large disk arrays," Algorithmica, vol. 12, no. 2-3, pp. 182-208, 1994.
|
| |
195
|
|
| |
196
|
|
| |
197
|
W.-K. Hon, T.-W. Lam, R. Shah, S.-L. Tam, and J. S. Vitter, "Cache-oblivious index for approximate string matching," in Proceedings of the Symposium on Combinatorial Pattern Matching, pp. 40-51, London, Ontario, Canada: Springer-Verlag, July 2007.
|
| |
198
|
W.-K. Hon, R. Shah, P. J. Varman, and J. S. Vitter, "Tight competitive ratios for parallel disk prefetching and caching," in Proceedings of the ACM Symposium on Parallel Algorithms and Architectures, Munich: ACM Press, June 2008.
|
 |
199
|
|
| |
200
|
D. Hutchinson, A. Maheshwari, J.-R. Sack, and R. Velicescu, "Early experiences in implementing the buffer tree," in Proceedings of the Workshop on Algorithm Engineering, Springer-Verlag, 1997.
|
| |
201
|
|
| |
202
|
|
 |
203
|
Piotr Indyk , Rajeev Motwani , Prabhakar Raghavan , Santosh Vempala, Locality-preserving hashing in multidimensional spaces, Proceedings of the twenty-ninth annual ACM symposium on Theory of computing, p.618-625, May 04-06, 1997, El Paso, Texas, United States
[doi> 10.1145/258533.258656]
|
 |
204
|
|
| |
205
|
|
 |
206
|
|
| |
207
|
|
| |
208
|
I. Kamel, M. Khalil, and V. Kouramajian, "Bulk insertion in dynamic R-trees," in Proceedings of the International Symposium on Spatial Data Handling , pp. 3B, 31-42, 1996.
|
| |
209
|
|
| |
210
|
|
| |
211
|
|
| |
212
|
J. Kärkkäinen and S. S. Rao, "Full-text indexes in external memory," in Algorithms for Memory Hierarchies, (U. Meyer, P. Sanders, and J. Sibeyn, eds.), ch. 7, pp. 149-170, Berlin: Springer-Verlag, 2003.
|
| |
213
|
I. Katriel and U. Meyer, "Elementary graph algorithms in external memory," in Algorithms for Memory Hierarchies, (U. Meyer, P. Sanders, and J. Sibeyn, eds.), ch. 4, pp. 62-84, Berlin: Springer-Verlag, 2003.
|
| |
214
|
R. Khandekar and V. Pandit, "Offline Sorting Buffers On Line," in Proceedings of the International Symposium on Algorithms and Computation, pp. 81-89, Springer-Verlag, December 2006.
|
| |
215
|
|
| |
216
|
|
| |
217
|
|
| |
218
|
|
| |
219
|
|
| |
220
|
|
| |
221
|
D. E. Knuth, MMIXware. Berlin: Springer-Verlag, 1999.
|
| |
222
|
D. E. Knuth, J. H. Morris, and V. R. Pratt, "Fast pattern matching in strings," SIAM Journal on Computing, vol. 6, pp. 323-350, 1977.
|
 |
223
|
George Kollios , Dimitrios Gunopulos , Vassilis J. Tsotras, On indexing mobile objects, Proceedings of the eighteenth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, p.261-272, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
[doi> 10.1145/303976.304002]
|
 |
224
|
|
| |
225
|
M. Kowarschik and C. Weiß, "An overview of cache optimizaiton techniques and cache-aware numerical algorithms," in Algorithms for Memory Hierarchies , (U. Meyer, P. Sanders, and J. Sibeyn, eds.), ch. 10, pp. 213-232, Berlin: Springer-Verlag, 2003.
|
| |
226
|
|
| |
227
|
|
| |
228
|
K. Küspert, "Storage utilization in B*-trees with a generalized overflow technique," Acta Informatica, vol. 19, pp. 35-55, 1983.
|
 |
229
|
|
| |
230
|
R. Laurini and D. Thompson, Fundamentals of Spatial Information Systems. Academic Press, 1992.
|
 |
231
|
|
| |
232
|
|
| |
233
|
|
| |
234
|
Z. Li, P. H. Mills, and J. H. Reif, "Models and resource metrics for parallel and distributed computation," Parallel Algorithms and Applications, vol. 8, pp. 35-59, 1996.
|
| |
235
|
|
| |
236
|
|
 |
237
|
|
 |
238
|
|
| |
239
|
|
| |
240
|
|
| |
241
|
|
| |
242
|
|
| |
243
|
A. Maheshwari and N. Zeh, "A survey of techniques for designing I/O-efficient algorithms," in Algorithms for Memory Hierarchies, (U. Meyer, P. Sanders, and J. Sibeyn, eds.), ch. 3, pp. 36-61, Berlin: Springer-Verlag, 2003.
|
| |
244
|
A. Maheshwari and N. Zeh, "I/O-optimal algorithms for outerplanar graphs," Journal of Graph Algorithms and Applications, vol. 8, pp. 47-87, 2004.
|
| |
245
|
V. Mäkinen, G. Navarro, and K. Sadakane, "Advantages of backward searching --efficient secondary memory and distributed implementation of compressed suffix arrays," in Proceedings of the International Symposium on Algorithms and Computation, pp. 681-692, Springer-Verlag, 2004.
|
| |
246
|
|
| |
247
|
|
| |
248
|
|
| |
249
|
Yossi Matias , Eran Segal , Jeffrey Scott Vitter, Efficient bundle sorting, Proceedings of the eleventh annual ACM-SIAM symposium on Discrete algorithms, p.839-848, January 09-11, 2000, San Francisco, California, United States
|
 |
250
|
|
| |
251
|
E. M. McCreight, "Priority Search Trees," SIAM Journal on Computing, vol. 14, pp. 257-276, May 1985.
|
| |
252
|
|
| |
253
|
|
| |
254
|
|
| |
255
|
U. Meyer, "On dynamic breadth-first search in external-memory," in Proceedings of the Symposium on Theoretical Aspects of Computer Science, (Schloss Dagstuhl, Germany), pp. 551-560, Internationales Begegnungs- und Forschungszentrum für Informatik, 2008.
|
| |
256
|
U. Meyer, "On trade-offs in external-memory diameter approximation," in Proceedings of the Scandinavian Workshop on Algorithm Theory, (Gothenburg, Sweden), Springer-Verlag, July 2008.
|
| |
257
|
|
| |
258
|
|
| |
259
|
|
| |
260
|
|
 |
261
|
|
| |
262
|
|
| |
263
|
J. K. Mullin, "Spiral storage: Efficient dynamic hashing with constant performance," The Computer Journal, vol. 28, pp. 330-334, July 1985.
|
| |
264
|
|
| |
265
|
|
| |
266
|
|
 |
267
|
|
 |
268
|
|
| |
269
|
J. Nievergelt and E. M. Reingold, "Binary search tree of bounded balance," SIAM Journal on Computing, vol. 2, pp. 33-43, March 1973.
|
| |
270
|
|
| |
271
|
M. H. Nodine, M. T. Goodrich, and J. S. Vitter, "Blocking for external graph searching," Algorithmica, vol. 16, pp. 181-214, August 1996.
|
| |
272
|
|
 |
273
|
|
 |
274
|
|
| |
275
|
|
 |
276
|
|
 |
277
|
|
| |
278
|
|
| |
279
|
|
 |
280
|
Hwee Hwa Pang , Michael J. Carey , Miron Livny, Partially preemptible hash joins, Proceedings of the 1993 ACM SIGMOD international conference on Management of data, p.59-68, May 25-28, 1993, Washington, D.C., United States
|
| |
281
|
|
 |
282
|
|
| |
283
|
|
| |
284
|
|
| |
285
|
O. Procopiuc, P. K. Agarwal, L. Arge, and J. S. Vitter, "Bkd-tree: A dynamic scalable kd-tree," in Proceedings of the International Symposium on Spatial and Temporal Databases, Santorini, Greece: Springer-Verlag, July 2003.
|
| |
286
|
S. J. Puglisi, W. F. Smyth, and A. Turpin, "Inverted files versus suffix arrays for locating patterns in primary memory," in Proceedings of the International Symposium on String Processing Information Retrieval, pp. 122-133, Springer-Verlag, 2006.
|
| |
287
|
N. Rahman and R. Raman, "Adapting radix sort to the memory hierarchy," in Workshop on Algorithm Engineering and Experimentation, Springer-Verlag, January 2000.
|
| |
288
|
|
 |
289
|
|
| |
290
|
|
 |
291
|
|
| |
292
|
|
 |
293
|
|
 |
294
|
|
| |
295
|
|
| |
296
|
|
 |
297
|
Simonas Šaltenis , Christian S. Jensen , Scott T. Leutenegger , Mario A. Lopez, Indexing the positions of continuously moving objects, Proceedings of the 2000 ACM SIGMOD international conference on Management of data, p.331-342, May 15-18, 2000, Dallas, Texas, United States
|
 |
298
|
|
| |
299
|
|
| |
300
|
|
 |
301
|
|
 |
302
|
|
| |
303
|
|
| |
304
|
P. Sanders, S. Egner, and J. Korst, "Fast concurrent access to parallel disks," Algorithmica, vol. 35, no. 1, pp. 21-55, 2002.
|
| |
305
|
|
| |
306
|
J. E. Savage and J. S. Vitter, "Parallelism in space-time tradeoffs," in Advances in Computing Research, (F. P. Preparata, ed.), pp. 117-146, JAI Press, 1987.
|
 |
307
|
Steven W. Schlosser , John Linwood Griffin , David F. Nagle , Gregory R. Ganger, Designing computer systems with MEMS-based storage, Proceedings of the ninth international conference on Architectural support for programming languages and operating systems, p.1-12, November 2000, Cambridge, Massachusetts, United States
|
| |
308
|
|
| |
309
|
|
| |
310
|
Margo Seltzer , Keith A. Smith , Hari Balakrishnan , Jacqueline Chang , Sara McMains , Venkata Padmanabhan, File system logging versus clustering: a performance comparison, Proceedings of the USENIX 1995 Technical Conference Proceedings on USENIX 1995 Technical Conference Proceedings, p.21-21, January 16-20, 1995, New Orleans, Louisiana
|
 |
311
|
|
 |
312
|
|
 |
313
|
|
 |
314
|
Elizabeth Shriver , Arif Merchant , John Wilkes, An analytic behavior model for disk drives with readahead caches and request reordering, Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems, p.182-191, June 22-26, 1998, Madison, Wisconsin, United States
|
| |
315
|
E. A. M. Shriver and M. H. Nodine, "An introduction to parallel I/O models and algorithms," in Input/Output in Parallel and Distributed Computer Systems , (R. Jain, J. Werth, and J. C. Browne, eds.), ch. 2, pp. 31-68, Kluwer Academic Publishers, 1996.
|
| |
316
|
|
| |
317
|
J. F. Sibeyn, "From parallel to external list ranking," Technical Report MPI-I-97-1-021, Max-Planck-Institut, September 1997.
|
| |
318
|
|
| |
319
|
|
| |
320
|
R. Sinha, S. Puglisi, A. Moffat, and A. Turpin, "Improving suffix array locality for fast pattern matching on disk," in Proceedings of the ACM SIGMOD International Conference on Management of Data, Vancouver: ACM Press, June 2008.
|
| |
321
|
|
 |
322
|
|
| |
323
|
|
| |
324
|
R. Tamassia and J. S. Vitter, "Optimal cooperative search in fractional cascaded data structures," Algorithmica, vol. 15, pp. 154-171, February 1996.
|
| |
325
|
"TerraServer-USA: Microsoft's online database of satellite images," Available on the World-Wide Web at http://terraserver.microsoft.com/.
|
| |
326
|
|
| |
327
|
"Topologically Integrated Geographic Encoding and Referencing system, TIGER/Line 1992 datafiles," Available on the World-Wide Web at http://www.census.gov/geo/www/tiger/, 1992.
|
| |
328
|
|
| |
329
|
L. Toma and N. Zeh, "I/O-efficient algorithms for sparse graphs," in Algorithms for Memory Hierarchies, (U. Meyer, P. Sanders, and J. Sibeyn, eds.), ch. 5, pp. 85-109, Berlin: Springer-Verlag, 2003.
|
| |
330
|
TPIE User Manual and Reference, "The manual and software distribution," available on the web at http://www.cs.duke.edu/TPIE/, 1999.
|
| |
331
|
J. D. Ullman and M. Yannakakis, "The input/output complexity of transitive closure," Annals of Mathematics and Artificial Intelligence, vol. 3, pp. 331-360, 1991.
|
 |
332
|
|
| |
333
|
|
| |
334
|
M. van Kreveld, J. Nievergelt, T. Roos, and P. Widmayer, eds., Algorithmic foundations of GIS. Vol. 1340 of Lecture Notes in Computer Science, Springer-Verlag, 1997.
|
| |
335
|
|
 |
336
|
|
| |
337
|
D. E. Vengroff and J. S. Vitter, "I/O-efficient scientific computation using TPIE," in Proceedings of NASA Goddard Conference on Mass Storage Systems , pp. II, 553-570, September 1996.
|
| |
338
|
P. Vettiger, M. Despont, U. Drechsler, U. Dürig, W. Häberle, M. I. Lutwyche, E. Rothuizen, R. Stutz, R. Widmer, and G. K. Binnig, "The 'Millipede' -- more than one thousand tips for future AFM data storage," IBM Journal of Research and Development, vol. 44, no. 3, pp. 323-340, 2000.
|
| |
339
|
|
| |
340
|
J. S. Vitter, Notes. 1999.
|
| |
341
|
|
 |
342
|
|
 |
343
|
|
| |
344
|
|
| |
345
|
J. S. Vitter and E. A. M. Shriver, "Algorithms for parallel memory I: Two-level memories," Algorithmica, vol. 12, no. 2-3, pp. 110-147, 1994.
|
| |
346
|
J. S. Vitter and E. A. M. Shriver, "Algorithms for parallel memory II: Hierarchical multilevel memories," Algorithmica, vol. 12, no. 2-3, pp. 148-169, 1994.
|
 |
347
|
|
 |
348
|
Jeffrey Scott Vitter , Min Wang , Bala Iyer, Data cube approximation and histograms via wavelets, Proceedings of the seventh international conference on Information and knowledge management, p.96-104, November 02-07, 1998, Bethesda, Maryland, United States
[doi> 10.1145/288627.288645]
|
| |
349
|
M. Wang, B. Iyer, and J. S. Vitter, "Scalable mining for classification rules in relational databases," in Herman Rubin Festschrift, Hayward, CA: Institute of Mathematical Statistics, Fall 2004.
|
| |
350
|
|
| |
351
|
|
| |
352
|
P. Weiner, "Linear pattern matching algorithm," in Proceedings of the IEEE Symposium on Switching and Automata Theory, pp. 1-11, 1973.
|
| |
353
|
|
 |
354
|
|
| |
355
|
|
 |
356
|
Ouri Wolfson , Prasad Sistla , Bo Xu , Jutai Zhou , Sam Chamberlain, DOMINO: databases fOr MovINg Objects tracking, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.547-549, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
|
| |
357
|
D. Womble, D. Greenberg, S. Wheat, and R. Riesen, "Beyond core: Making parallel computer I/O practical," in Proceedings of the DAGS Symposium on Parallel Computation, pp. 56-63, June 1993.
|
| |
358
|
|
 |
359
|
Yuni Xia , Sunil Prabhakar , Shan Lei , Reynold Cheng , Rahul Shah, Indexing continuously changing data with mean-variance tree, Proceedings of the 2005 ACM symposium on Applied computing, March 13-17, 2005, Santa Fe, New Mexico
[doi> 10.1145/1066677.1066932]
|
| |
360
|
A. C. Yao, "On random 2-3 trees," Acta Informatica, vol. 9, pp. 159-170, 1978.
|
| |
361
|
|
| |
362
|
|
| |
363
|
|
| |
364
|
|
| |
365
|
J. Ziv and A. Lempel, "Compression of individual sequences via variable-rate coding," IEEE Transactions on Information Theory, vol. 24, pp. 530-536, September 1978.
|
|