Home > Erasure Coding > Erasure Coding for Big Data

Erasure Coding for Big Data

References

  • [1] “Data center storage capacity worldwide,” https://www.statista.com/statistics/638593/worldwide-data-center-storage-capacity-cloud-vs-traditional/, accessed: February 16, 2019.
  • [2] “The NSA Data: Where Does It Go?” https://news.nationalgeographic.com/news/2013/06/130612-nsa-utah-data-center-storage-zettabyte-snowden/, accessed: February 16, 2019.
  • [3] S. B. Balaji, M. N. Krishnan, M. Vajha, V. Ramkumar, B. Sasidharan, and P. V. Kumar, “Erasure coding for distributed storage: an overview,” SCIENCE CHINA Information Sciences, vol. 61, no. 10, pp. 100 301:1–100 301:45, 2018.
  • [4] “Hadoop,” http://hadoop.apache.org.
  • [5] I. S. Reed and G. Solomon, “Polynomial codes over certain finite fields,” Journal of the society for industrial and applied mathematics, vol. 8, no. 2, pp. 300–304, 1960.
  • [6] F. J. MacWilliams and N. J. A. Sloane, The theory of error-correcting codes. Elsevier, 1977, vol. 16.
  • [7] M. Sathiamoorthy, M. Asteris, D. S. Papailiopoulos, A. G. Dimakis, R. Vadali, S. Chen, and D. Borthakur, “XORing Elephants: Novel Erasure Codes for Big Data,” PVLDB, vol. 6, no. 5, pp. 325–336, 2013.
  • [8] K. V. Rashmi, N. B. Shah, D. Gu, H. Kuang, D. Borthakur, and K. Ramchandran, “A Solution to the Network Challenges of Data Recovery in Erasure-coded Distributed Storage Systems: A Study on the Facebook Warehouse Cluster,” in Proc. 5th USENIX Workshop on Hot Topics in Storage and File Systems, 2013.
  • [9] A. Dimakis, P. Godfrey, Y. Wu, M. Wainwright, and K. Ramchandran, “Network coding for distributed storage systems,” IEEE Trans. Inf. Theory, vol. 56, no. 9, pp. 4539–4551, 2010.
  • [10] P. Gopalan, C. Huang, H. Simitci, and S. Yekhanin, “On the Locality of Codeword Symbols,” IEEE Trans. Inf. Theory, vol. 58, no. 11, pp. 6925–6934, 2012.
  • [11] K. Shanmugam, D. S. Papailiopoulos, A. G. Dimakis, and G. Caire, “A repair framework for scalar MDS codes,” IEEE J. Sel. Areas Commun., vol. 32, no. 5, pp. 998–1007, 2014.
  • [12] V. Guruswami and M. Wootters, “Repairing Reed-Solomon Codes,” IEEE Trans. Inf. Theory, vol. 63, no. 9, pp. 5684–5698, 2017.
  • [13] H. Dau, I. M. Duursma, H. M. Kiah, and O. Milenkovic, “Repairing Reed-Solomon Codes With Multiple Erasures,” IEEE Trans. Inf. Theory, vol. 64, no. 10, pp. 6567–6582, 2018.
  • [14] K. V. Rashmi, N. B. Shah, and K. Ramchandran, “A Piggybacking Design Framework for Read-and Download-Efficient Distributed Storage Codes,” IEEE Trans. Inf. Theory, vol. 63, no. 9, pp. 5802–5820, 2017.
  • [15] A. S. Rawat, I. Tamo, V. Guruswami, and K. Efremenko, “$\epsilon$-MSR codes with small sub-packetization,” in 2017 IEEE International Symposium on Information Theory, ISIT 2017, Aachen, Germany, June 25-30, 2017, 2017, pp. 2043–2047.
  • [16] Y. Hu, Y. Xu, X. Wang, C. Zhan, and P. Li, “Cooperative recovery of distributed storage systems from multiple losses with network coding,” IEEE Journal on Selected Areas in Communications, vol. 28, no. 2, pp. 268–276, 2010.
  • [17] S. El Rouayheb and K. Ramchandran, “Fractional repetition codes for repair in distributed storage systems,” in 48th Annual Allerton Conference on Communication, Control, and Computing, 2010, pp. 1510–1517.
  • [18] R. Ahlswede, N. Cai, S. R. Li, and R. W. Yeung, “Network information flow,” IEEE Trans. Inf. Theory, vol. 46, no. 4, pp. 1204–1216, 2000.
  • [19] K. V. Rashmi, N. B. Shah, P. V. Kumar, and K. Ramchandran, “Explicit construction of optimal exact regenerating codes for distributed storage,” in Proc. 47th Annu. Allerton Conf. Communication, Control, and Computing, Urbana-Champaign, IL, Sep. 2009, pp. 1243–1249.
  • [20] S. B. Balaji and P. V. Kumar, “A tight lower bound on the sub- packetization level of optimal-access MSR and MDS codes,” in Proc. IEEE Int. Symp. Inf. Theory, 2018, pp. 2381–2385.
  • [21] K. V. Rashmi, N. B. Shah, and P. V. Kumar, “Optimal Exact-Regenerating Codes for Distributed Storage at the MSR and MBR Points via a Product-Matrix Construction,” IEEE Trans. Inf. Theory, vol. 57, no. 8, pp. 5227–5239, 2011.
  • [22] M. Ye and A. Barg, “Explicit Constructions of Optimal-Access MDS Codes With Nearly Optimal Sub-Packetization,” IEEE Trans. Inf. Theory, vol. 63, no. 10, pp. 6307–6317, 2017.
  • [23] B. Sasidharan, M. Vajha, and P. V. Kumar, “An Explicit, Coupled-Layer Construction of a High-Rate MSR Code with Low Sub-Packetization Level, Small Field Size and All-Node Repair,” CoRR, vol. abs/1607.07335, 2016.
  • [24] C. Tian, J. Li, and X. Tang, “A generic transformation for optimal repair bandwidth and rebuilding access in MDS codes,” in Proc. Int. Symp. Inf. Theory. IEEE, 2017, pp. 1623–1627.
  • [25] M. Vajha, V. Ramkumar, B. Puranik, G. R. Kini, E. Lobo, B. Sasidharan, P. V. Kumar, A. Barg, M. Ye, S. Narayanamurthy, S. Hussain, and S. Nandi, “Clay codes: Moulding MDS codes to yield an MSR code,” in Proc. 16th USENIX Conference on File and Storage Technologies, 2018, pp. 139–154.
  • [26] “Ceph,” https://ceph.com/.
  • [27] “Clay code documentation in ceph,” http://docs.ceph.com/docs/nautilus/rados/operations/erasure-code-clay/.
  • [28] C. Huang, M. Chen, and J. Li, “Pyramid codes: Flexible schemes to trade space for access efficiency in reliable data storage systems,” in Proc. Sixth IEEE Int. Symp. Netw. Comput. Applications, 2007, pp. 79–86.
  • [29] C. Huang, H. Simitci, Y. Xu, A. Ogus, B. Calder, P. Gopalan, J. Li, and S. Yekhanin, “Erasure coding in windows azure storage,” in Proc. USENIX Annual Technical Conference, 2012, pp. 15–26.
  • [30] “Microsoft research blog: A better way to store data,” https://www.microsoft.com/en-us/research/blog/better-way-store-data/.
  • [31] I. Tamo and A. Barg, “A Family of Optimal Locally Recoverable Codes,” IEEE Trans. Inf. Theory, vol. 60, no. 8, pp. 4661–4676, 2014.
  • [32] B. Sasidharan, G. K. Agarwal, and P. V. Kumar, “Codes with hierarchical locality,” in Proc. IEEE Int. Symp. Inf. Theory, June 2015, pp. 1257–1261.
  • [33] A. Wang and Z. Zhang, “Repair Locality With Multiple Erasure Tolerance,” IEEE Trans. Inf. Theory, vol. 60, no. 11, pp. 6979–6987, 2014.
  • [34] A. Wang, Z. Zhang, and M. Liu, “Achieving arbitrary locality and availability in binary codes,” in Proc. IEEE International Symposium on Information Theory, Hong Kong, 2015, 2015, pp. 1866–1870.
  • [35] N. Prakash, V. Lalitha, and P. V. Kumar, “Codes with locality for two erasures,” in Proc. IEEE Int. Symp. Inf. Theory, 2014, pp. 1962–1966.
  • [36] S. B. Balaji, G. R. Kini, and P. V. Kumar, “A tight rate bound and a matching construction for locally recoverable codes with sequential recovery from any number of multiple erasures,” in Proc. IEEE Int. Symp. Inf. Theory, 2017, pp. 1778–1782.
  • [37] G. M. Kamath, N. Prakash, V. Lalitha, and P. V. Kumar, “Codes with local regeneration and erasure correction,” IEEE Trans. Inf. Theory, vol. 60, no. 8, pp. 4637–4660, 2014.
  • [38] A. S. Rawat, O. O. Koyluoglu, N. Silberstein, and S. Vishwanath, “Optimal Locally Repairable and Secure Codes for Distributed Storage Systems,” IEEE Trans. Inf. Theory, vol. 60, no. 1, pp. 212–236, 2014.
  • [39] M. N. Krishnan, R. A. Narayanan, and P. V. Kumar, “Codes with combined locality and regeneration having optimal rate, $d_{min}$ and linear field size,” in Proc. IEEE Int. Symp. Inf. Theory, 2018, pp. 1196–1200.
Pages ( 6 of 6 ): « Previous1 ... 45 6

Leave a Comment:

Your email address will not be published. Required fields are marked *