Multi-fidelity storage

ABSTRACT

Imaging sensors are inherently high bandwidth devices, and applications which store image data often encounter disk or memory limits. Commonly, upon reaching such a limit, storage systems will cease sampling or overwrite existing data in an oldest-first fashion. For most applications, neither approach is optimal. We introduce a flexible, policy-based, on-line algorithm for maximizing the application-specific value of data retained on a read/write/erase storage medium such as a hard disk or flash card. Our algorithm supports both selective discard of image frames, and selective lossless and lossy transformations which may retain information valuable to an application while reducing data storage requirements. We demonstrate that our multi-fidelity storage system can fairly manage a limited data storage medium on behalf of one or more image storage applications.

REFERENCES

• 1: Jon C. R. Bennett , Hui Zhang, Hierarchical packet fair queueing algorithms, IEEE/ACM Transactions on Networking (TON), v.5 n.5, p.675-689, Oct. 1997 [doi>10.1109/90.649568]
• 2: Michael J. Carey , David J. DeWitt , Michael J. Franklin , Nancy E. Hall , Mark L. McAuliffe , Jeffrey F. Naughton , Daniel T. Schuh , Marvin H. Solomon , C. K. Tan , Odysseas G. Tsatalos , Seth J. White , Michael J. Zwilling, Shoring up persistent applications, Proceedings of the 1994 ACM SIGMOD international conference on Management of data, p.383-394, May 24-27, 1994, Minneapolis, Minnesota, United States
• 3: A. Demers , S. Keshav , S. Shenker, Analysis and simulation of a fair queueing algorithm, Symposium proceedings on Communications architectures & protocols, p.1-12, September 25-27, 1989, Austin, Texas, United States
• 4: Michael R. Garey , David S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, W. H. Freeman & Co., New York, NY, 1979
• 5: P. B. Gibbons, B. Karp, Y. Ke, S. Nath, and S. Seshan. Irisnet: An architecture for a woldwide sensor web. IEEE Pervasive Computing, 2(4), 2003.
• 6: Larry Huston , Rahul Sukthankar , Rajiv Wickremesinghe , M. Satyanarayanan , Gregory R. Ganger , Erik Riedel , Anastassia Ailamaki, Diamond: A Storage Architecture for Early Discard in Interactive Search, Proceedings of the 3rd USENIX Conference on File and Storage Technologies, March 31-31, 2004, San Francisco, CA
• 7: Samuel Madden , Mehul Shah , Joseph M. Hellerstein , Vijayshankar Raman, Continuously adaptive continuous queries over streams, Proceedings of the 2002 ACM SIGMOD international conference on Management of data, June 03-06, 2002, Madison, Wisconsin [doi>10.1145/564691.564698]
• 8: M. Mesnier, G. R. Ganger, and E. Riedel. Object-based storage. IEEE Communications Magazine, 41(8):84--90, 2003.
• 9: Mitsubishi Imaging Products. HS-S9600U Time-lapse Video Recorder Datasheet, 2004.
• 10: Moving Picture Experts Group. ISO/IEC 14496: MPEG-4 Standard, 1999.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.2 Information Storage

Additional Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.2 Storage Management

General Terms:
Algorithms, Design, Management

Keywords:
variable fidelity, video surveillance, weighted fair-share storage