An Open Source Virtual Machine Monitor Framework For Modern Architectures

Do you program or manage programming in any capacity? Please take our floating point survey.

About

The V3VEE project (v3vee.org) creates kernel and virtual machine monitor frameworks for modern architectures that permit the compile-time and run-time creation of OS and run-time environments with different structures, including those optimized for computer architecture research, parallelism research, and use in high performance computing. V3VEE began as an NSF-funded collaborative project between Northwestern University and the University of New Mexico. It expanded to include five DOE-funded partner institutions: Northwestern University, the University of New Mexico, the University of Pittsburgh, Sandia National Laboratories, and Oak Ridge National Laboratory. It currently is supported by the DOE's Hobbes Project (which involves 15 academic and national labs partners) and the NSF. V3VEE is a community resource development effort that anyone can contribute to.

Active Efforts

Talks

  • Paths to Fast Barrier Synchronization on the Node, Conor Hetland, HPDC 2019 (pdf, pdf poster)
  • An Evaluation of Asynchronous Software Events on Modern Hardware, Kyle Hale, MASCOTS 2018
  • Hard Real-time Scheduling for Parallel Run-time Systems, Peter Dinda, HPDC 2018 (pdf)
  • Dark Shadows: User-level Guest/Host Linux Process Shadowing, Peter Dinda, IC2E 2017 (pptx)
  • Automatic Hybridization of Runtime Systems, Kyle Hale, HPDC 2016 (pdf)
  • Multiverse: Automatic Hybridization of Runtime Systems, Kyle Hale and Conor Hetland, GCASR 2016 (poster pdf)
  • Enabling Hybrid Parallel Runtimes Through Kernel and Virtualization Support, Kyle Hale, VEE 2016 (pdf)
  • A Case For Transforming Parallel Runtimes into Operating System Kernels, Kyle Hale, HPDC 2015 (pdf)
  • Guarded Modules: Adaptively Extending the VMM's Privileges Into the Guest, Kyle Hale, ICAC 2014 (pdf)
  • ConCORD: Easily Exploiting Memory Content Redundancy Through the Content-aware Service Command, Lei Xia, HPDC 2014 (pptx)
  • VMM Emulation of Intel Hardware Transactional Memory, Maciej Swiech, ROSS 2014 (pdf)
  • Virtual TCP Offload: Optimizing Ethernet Overlay Performance on Advanced Interconnects, Zheng Cui, HPDC 2013 (pdf)
  • Optimizing Overlay-based Virtual Networking Through Opportunistic Interrupts and Cut-Through Forwarding, Zheng Cui, SC 2012 (pdf)
  • A Case for Dual Stack Virtualization: Consolidating HPC and Commodity Applications in the Cloud, Jack Lange, SOCC 2012 (pptx)
  • Shifting GEARS to Enable Guest-context Virtual Services, Kyle Hale, ICAC 2012 (pdf)
  • Better than Native: Using Virtualization to Improve Compute Node Performance, Brian Kocoloski, ROSS 2012 (pdf)
  • VNET/P: Bridging the Cloud and High Performance Computing Through Fast Overlay Networking, Lei Xia, HPDC 2012 (Best Paper Nominee) (pdf-talk, pdf-poster)
  • Dynamic Adaptive Virtual Core Mapping to Improve Power, Energy, and Performance in Multi-socket Multicores, Chang Bae, HPDC 2012 (pdf-talk, pdf-poster)
  • A Case for Tracking and Exploiting Inter-node and Intra-node Memory Content Sharing in Virtualized Large-Scale Parallel Systems , Lei Xia, VTDC 2012 (pdf)
  • Enhancing Virtualized Application Performance Through Dynamic Adaptive Paging Mode Selection, Chang Bae, ICAC 2011 (pdf)
  • SymCall: Symbiotic Virtualization Through VMM-to-Guest Upcalls, Jack Lange, VEE 2011 (pptx)
  • Virtual WiFi: Bringing Virtualization From Wired to Wireless, Lei Xia, VEE 2011 (pptx) (work done while Lei Xia was on internship at Intel)
  • Minimal Overhead Virtualization of a Large Scale Supercomputer, Jack Lange, VEE 2011 (pptx)
  • Opportunities for Leveraging OS Virtualization in High-End Supercomputing, Kevin Pedretti, MASVDC 2010 (pdf)
  • Back to the Futures: Incremental Parallelization of Existing Sequential Runtime Systems, James Swaine, OOPSLA 2010 (pdf)
  • An Introduction to the V3VEE Project and the Palacios Virtual Machine Monitor, Peter Dinda, Keynote Talk, VTDC 2010
  • Palacios and Kitten: New High Performance Operating Systems for Scalable Virtualized and Native Supercomputing, Jack Lange, IPDPS 2010 (ppt)
  • Towards Virtual Passthrough I/O on Commodity Devices, Lei Xia, WIOV 2008 (ppt)

    • Papers

  • C. Hetland, G. Tziantzioulis, B. Suchy, K. Hale, N. Hardavellas, P. Dinda, Prospects for Functional Address Translation, Proceedings of the 27th IEEE International Conference on the Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS 2019), October, 2019. To Appear.
  • C. Hetland, G. Tziantzioulis, B. Suchy, M. Leonard, J. Han, J. Albers, N. Hardavellas, P. Dinda, Paths to Fast Barrier Synchronization on the Node, Proceedings of the 28th ACM Symposium on High-performance Parallel and Distributed Computing (HPDC 2019), June, 2019. pdf
  • K. Hale, P. Dinda, An Evaluation of Asynchronous Software Events on Modern Hardware, Proceedings of the 26th IEEE International Symposium on the Modeling, Analysis, and Simulaton of Computer and Telecommunication Systems (MASCOTS 2018), September, 2018. pdf
  • P. Dinda, X. Wang, J. Wang, C. Beauchene, C. Hetland, Hard Real-time Scheduling for Parallel Run-time Systems, Proceedings of the 27th ACM Symposium on High-performance Parallel and Distributed Computing (HPDC 2018), June, 2018. pdf
  • P. Dinda, A. Guliani, Dark Shadows: User-level Guest/Host Linux Process Shadowing, Proceedings of the 5th IEEE International Symposium on Cloud Engineering (IC2E 2017), April, 2017. pdf (Best Paper)
  • K.Hale, Hybrid Runtime Systems, Doctoral Dissertation, Northwestern University. Published as Technical Report NWU-EECS-16-12, Department of Electrical Engineering and Computer Science, Northwestern University, August, 2016. pdf
  • K. Hale, C. Hetland, P. Dinda, Automatic Hybridization of Runtime Systems, Proceedings of the 25th ACM Symposium on High-performance Parallel and Distributed Computing (HPDC 2016), June, 2016. pdf
  • K. Hale, P. Dinda, Enabling Hybrid Parallel Runtimes Through Kernel and Virtualization Support, Proceedings of the 12th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2016), April, 2016. pdf
  • P. Dinda, A. Guliani, Dark Shadows: User-level Guest/Host Linux Process Shadowing, Technical Report NWU-EECS-16-04, Department of Electrical Engineering and Computer Science, Northwestern University, March, 2016. pdf
  • K. Hale, P. Dinda, Pushing Software Events to the Hardware Limit, Technical Report NWU-EECS-16-2, Department of Electrical Engineering and Computer Science, Northwestern University, March, 2016. pdf
  • K. Hale, P. Dinda, A Case for Transforming Parallel Runtime Systems Into Operating System Kernels, Proceedings of the 24th ACM Symposium on High-performance Parallel and Distributed Computing (HPDC 2015), June, 2015. pdf
  • K. Hale, P. Dinda, Details of the Case for Transforming Parallel Runtime Systems Into Operating System Kernels, Technical Report NWU-EECS-15-1, Department of Electrical Engineering and Computer Science, Northwestern University, April, 2015. pdf
  • K. Hale, P. Dinda, Guarded Modules: Adaptively Extending the VMM's Privileges Into the Guest, Proceedings of the 11th International Conference on Autonomic Computing (ICAC 2014), June, 2014. pdf
  • L. Xia, K. Hale, P. Dinda, ConCORD: Easily Exploiting Memory Content Redundancy Through the Content-aware Service Command, Proceedings of the 23rd ACM Symposium on High-performance Parallel and Distributed Computing (HPDC 2014), June, 2014. pdf
  • M. Swiech, K. Hale, P. Dinda, VMM Emulation of Intel Hardware Transactional Memory, Proceedings of the 4th International Workshop on Runtime and Operating Systems for Supercomputers (ROSS 2014), June, 2014. pdf
  • C. Bae, Dynamic Adaptive Resource Management in a Virtualized NUMA Multicore System for Optimizing Power, Energy, and Performance, Doctoral Dissertation, Northwestern University. Published as Technical Report NWU-EECS-13-07, Department of Electrical Engineering and Computer Science, Northwestern University, July, 2013. pdf
  • Z. Cui, Enhancing HPC on Virtual Systems in Clouds through Optimizing Virtual Overlay Networks, Doctoral Dissertation, Department of Computer Science, University of New Mexio, July, 2013. pdf
  • L. Xia, ConCORD: Tracking and Exploiting Cross-Node Memory Content Redundancy in Large-Scale Parallel Systems, Doctoral Dissertation, Northwestern University. Published as Technical Report NWU-EECS-13-05, Department of Electrical Engineering and Computer Science, Northwestern University, July, 2013. pdf
  • L. Xia, Z. Cui, J. Lange, Y. Tang, P. Dinda, P. Bridges, Fast VMM-based Overlay Networking For Bridging the Cloud and High Performance Computing, Cluster Computing, Accepted, To Appear. pdf
  • R. Brightwell, R. Oldfield, D. Bernholdt, A. Maccabe, E. Brewer, P. Bridges, P. Dinda, J. Dongarra, C. Iancu, M. Lang, J. Lange, D. Lowenthal, F. Mueller, K. Schwan, T. Sterling and P. Teller, Hobbes: Composition and Virtualization as the Foundations of an Extreme-scale OS/R, Proceedings of the 3rd International Workshop on Runtime and Operating Systems for Supercomputers (ROSS 2013), June, 2013. pdf
  • Z. Cui, P. Bridges, J. Lange, P. Dinda, Virtual TCP Offload: Optimizing Ethernet Overlay Performance on Advanced Interconnects, Proceedings of the 22nd ACM Symposium on High-performance Parallel and Distributed Computing (HPDC 2013), June, 2013. pdf
  • K. Hale, P. Dinda, Guarded Execution of Privileged Code in the Guest, Technical Report NWU-EECS-13-04, Department of Electrical Engineering and Computer Science, Northwestern University, April, 2013. pdf
  • M. Swiech, K. Hale, P. Dinda, VMM-based Emulation of Intel Hardware Transactional Memory, Technical Report NWU-EECS-13-03, Department of Electrical Engineering and Computer Science, Northwestern University, April, 2013. pdf
  • Z. Cui, L. Xia, P. Bridges, P. Dinda, J. Lange, Optimizing Overlay-based Virtual Networking Through Optimistic Interrupts and Cut-through Forwarding, Proceedings of the ACM/IEEE International Conference on High Performance Computing, Networking, Storage and Analysis (SC'12) (Supercomputing), November, 2012. pdf
  • B. Kocoloski, J. Ouyang, and J. Lange, A Case for Dual Stack Virtualization: Consolidating HPC and Commodity Applications in the Cloud, Proceedings of the third ACM Symposium on Cloud Computing, (SOCC 2012), October, 2012. pdf
  • K. Hale, L. Xia, P. Dinda, Shifting GEARS to Enable Guest-context Virtual Services, Proceedings of the 9th International Conference on Autonomic Computing (ICAC 2012), September, 2012. pdf
  • B. Kocoloski, J. Lange, Better than Native: Using Virtualization to Improve Compute Node Performance, Proceedings of the 2nd International Workshop on Runtime and Operating Systems for Supercomputers (ROSS 2012), June, 2012. pdf
  • L. Xia, Z. Cui, J. Lange, Y. Tang, P. Dinda, P. Bridges, VNET/P: Bridging the Cloud and High Performance Computing Through Fast Overlay Networking, Proceedings of the 21st ACM Symposium on High-performance Parallel and Distributed Computing (HPDC 2012), June, 2012. (Best Paper Nominee) pdf
  • C. Bae, L. Xia, P. Dinda, J. Lange, Dynamic Adaptive Virtual Core Mapping to Improve Power, Energy, and Performance in Multi-socket Multicores, Proceedings of the 21st ACM Symposium on High-performance Parallel and Distributed Computing (HPDC 2012), June, 2012. pdf
  • L. Xia, P. Dinda, A Case for Tracking and Exploiting Inter-node and Intra-node Memory Content Sharing in Virtualized Large-Scale Parallel Systems, Proceedings of the 6th International Workshop on Virtualization Technologies in Distributed Computing (VTDC 2012), June, 2012. pdf
  • P. Bridges, D. Arnold, K. Pedretti, M. Suresh, F. Lu, P. Dinda, R. Joseph, J. Lange, Virtual Machine-based Emulation of Future Generation High-performance Computing Systems, International Journal of High Performance Computing Applications, Volume 26, Number 2, pages 125-135, May, 2012. pdf
  • 唐源, 夏磊, 崔峥, J. Lange, P. Dinda, P. Bridges, 李建平, 嵌入虚拟机监视器的高性能虚拟网络, 仪器仪表学报, 2012, 33(5): 1195-1199 (English: Y. Tang, L. Xia, Z. Cui, J. Lange, P. Dinda, P. Bridges, J. Li, High Performance Virtual Network Embedding Virtual Machine Monitor, Chinese Journal of Scientific Instrument, Volume 33, Number 5, pages 1195-1199, May, 2012.) pdf
  • J. Lange, P. Dinda, K. Hale, L. Xia, An Introduction to the Palacios Virtual Machine Monitor---Version 1.3, Technical Report NWU-EECS-11-10, Department of Electrical Engineering and Computer Science, Northwestern University, November, 2011. pdf 2c1;2c
  • C. Bae, J. Lange, P. Dinda, Enhancing Virtualized Application Performance through Dynamic Adaptive Paging Mode Selection, Proceedings of the 8th International Conference on Autonomic Computing (ICAC 2011), June, 2011. pdf
  • J. Lange, P. Dinda, SymCall: Symbiotic Virtualization Through VMM-to-Guest Upcalls, Proceedings of the 2011 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2011), March, 2011. pdf
  • J. Lange, K. Pedretti, P. Dinda, P. Bridges, C. Bae, P. Soltero, A. Merritt, Minimal Overhead Virtualization of a Large Scale Supercomputer, Proceedings of the 2011 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2011), March, 2011. pdf
  • L. Xia, S. Kumar, X. Yang, P. Gopalakrishnan, Y. Liu, S. Schoenberg, X. Guo, Virtual WiFi: Bringing Virtualization from Wired to Wireless, Proceedings of the 2011 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2011), March, 2011. pdf (work done while L. Xia was on internship at Intel)
  • J. Lange, Symbiotic Virtualization, Doctoral Dissertation, Northwestern University. Published as Technical Report NWU-EECS-10-08, Department of Electrical Engineering and Computer Science, Northwestern University, September, 2010. pdf
  • J. Swaine, K. Tew, P. Dinda, R. Findler, M. Flatt, Back to the Futures: Incremental Parallelization of Existing Sequential Runtime Systems, Proceedings of the ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 2010), October, 2010. pdf
  • G. Hoang, C. Bae, J. Lange, L. Zhang, P. Dinda, R. Joseph, A Case for Alternative Nested Paging Models for Virtualized Systems, Computer Architecture Letters, Volume 9, Number 1, January-June, 2010. pdf
  • J. Lange, K. Pedretti, T. Hudson, P. Dinda, Z. Cui, L. Xia, P. Bridges, A. Gocke, S. Jaconette, M. Levenhagen, R. Brightwell, Palacios and Kitten: New High Performance Operating Systems for Scalable Virtualized and Native Supercomputing, Proceedings of the 24th IEEE International Parallel and Distributed Processing Symposium (IPDPS 2010), April, 2010. pdf
  • C. Bae, J. Lange, P. Dinda, Comparing Approaches to Virtualized Page Translation in Modern VMMs, Technical Report NWU-EECS-10-07, Department of Electrical Engineering and Computer Science, Northwestern University, April, 2010. pdf
  • J. Swaine, Scheme With Futures: Incremental Parallelization in a Language Virtual Machine, Masters Thesis, Technical Report NWU-EECS-09-20, Department of Electrical Engineering and Computer Science, Northwestern University, December, 2009. pdf
  • J. Lange, K. Pedretti, T. Hudson, P. Dinda, Z. Cui, L. Xia, P. Bridges, S. Jaconette, M. Levenhagen, R. Brightwell, P. Widener, Palacios and Kitten: High Performance Operating Systems For Scalable Virtualized and Native Supercomputing, Technical Report NWU-EECS-09-14, Department of Electrical Engineering and Computer Science, Northwestern University, July, 2009. pdf
  • L. Xia, J. Lange, P. Dinda, and C. Bae, Investigating Virtual Passthrough I/O on Commodity Devices, Operating Systems Review, Volume 43, Number 3, July, 2009. pdf
  • J. Lange, P. Dinda, An Introduction to the Palacios Virtual Machine Monitor---Release 1.0, Technical Report NWU-EECS-08-11, Department of Electrical Engineering and Computer Science, Northwestern University. pdf
  • L. Xia, J. Lange, and P. Dinda, Towards Virtual Passthrough I/O on Commodity Devices, Proceedings of the First Workshop on I/O Virtualization at OSDI (WIOV), pdf talk (pdf)

    • Teaching

  • Peter Dinda at Northwestern will teach a new course in low-level software development in Winter Quarter 2016.
  • Peter Dinda at Northwestern taught EECS 441 (Resource Virtualization) in Winter Quarter 2015.
  • Peter Dinda at Northwestern taught EECS 441 (Resource Virtualization) in Winter Quarter 2014.
  • Peter Dinda at Northwestern taught EECS 441 (Resource Virtualization) in Winter Quarter 2013.
  • Peter Dinda at Northwestern taught EECS 441 (Resource Virtualization) in Spring Quarter 2012.
  • Peter Dinda at Northwestern taught EECS 441 (Resource Virtualization) in Winter Quarter 2011.
  • Peter Dinda at Northwestern taught INF_TECH 490 (Resource Virtualization and the Enterprise) in Spring, 2010.
  • Peter Dinda at Northwestern taught EECS 441 (Resource Virtualization) in Winter Quarter 2010.
  • Peter Dinda at Northwestern taught EECS 441 (Resource Virtualization) during Winter Quarter 2009.
  • Jack Lange at Northwestern taught MSIT 491 (Resource Virtualization and the Enterprise) during Winter Quarter 2009.
  • Peter Dinda at Northwestern taught MSIT 491 (Resource Virtualization and the Enterprise) during Winter Quarter 2007.

    • The Team

  • Peter Dinda, PI at Northwestern University
  • Patrick Bridges, PI at the University of New Mexico
  • Kevin Pedretti, PI at Sandia National Laboratories
  • Stephen Scott, PI at Oak Ridge National Laboratory
  • Jack Lange, PI at the University of Pittsburgh
  • Fabian Bustamante, co-PI at Northwestern University
  • Russ Joseph, co-PI at Northwestern University
  • Dorian Arnold, co-PI at the University of New Mexico
  • Barney Maccabe, co-PI at the University of New Mexico
  • Lei Xia, Ph.D. student at Northwestern University
  • Chang Bae, Ph.D. student at Northwestern University
  • Kyle Hale, Ph.D. student at Northwestern University
  • Maciej Swiech, Ph.D. student at Northwestern University
  • Giang Hoang, Ph.D. student at Northwestern University
  • James Swaine, Ph.D. student at Northwestern University (primarily affiliated with Racket)
  • Feng Lu, Ph.D. student at Northwestern University
  • Zheng Cui, Ph.D. student at the University of New Mexico
  • Brian Kocoloski, Ph.D. student at the University of Pittsburgh
  • Trammell Hudson, Researcher at Sandia National Labs
  • Thomas Naughton, Researcher at Oak Ridge National Lab
  • Geoffroy Vallee, Researcher at Oak Ridge National Lab
  • John Hudson, undergraduate student at Northwestern University (NSF REU)
  • Steven Jaconette, undergraduate student at Northwestern University (NSF REU)
  • Andy Gocke, undergraduate student at Northwestern University (NSF REU)
  • Mat Wojcik, undergraduate student at Northwestern University (NSF REU)
  • Madhav Suresh, undergraduate student at Northweestern University (NSF REU)
  • Brad Weinberger, undergraduate student at Northwestern University (NSF REU)
  • Peter Kamm, undergraduate student at Northwestern University (NSF REU)
  • Rumou Duan, undergraduate student at Northwestern University
  • Robert Deloatch, undergraduate student at University of Maryland (Northwestern SROP student)
  • Philip Soltero, undergraduate student at the University of New Mexico
  • Nathan Graham, undergraduate student at the University of New Mexico
  • Yuan Tang, visiting scholar at Northwestern University
  • Patrick Widener, research assistant professor at the University of New Mexico
    • We are continuously looking for people to become engaged in this project. There are numerous ways to do so:
  • We are looking for graduate students at Northwestern University, the University of New Mexico, and the University of Pittsburgh.
  • We have independent study and paid REU opportunities for undergraduate students at Northwestern University, the University of New Mexico, and the University of Pittburgh.
  • This is an open source community development project and we encourage involvement by the broader community.

    • Related Projects

  • Virtuoso Project at Northwestern
  • Kitten HPC OS at Sandia National Labs
  • Catamount series of HPC OSes at the University of New Mexico and Sandia National Labs
  • Prescience Lab Testbed (password protected)

    • Acknowledgments

    This project is made possible by support from the National Science Foundation (NSF) via grants CCF-1533560 (Northwestern), CNS-0709168 (Northwestern), CNS-0707365 (UNM), the Department of Energy (DOE) via grant DE-SC0005343 (Northwestern, UNM, U.Pittsburgh, Sandia, and ORNL), and Sandia National Laboratories through the Hobbes Project, which is funded by the 2013 Exascale Operating and Runtime Systems Program under the Office of Advanced Scientific Computing Research in the DOE Office of Science. Seed funding to help start the development of Palacios at Northwestern was provided via a subcontract from Oak Ridge National Laboratory (ORNL) on DOE grant DE-AC05-00OR22725. Seed funding to help start multicore guest development at Northwestern was provided via a subcontract from Sandia National Laboratories. Jack Lange was partially supported by a Symantec Research Labs Fellowship. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-AC04-94AL85000.