VMware Experts Program Big Data
This week VMware is holding the VMware Experts Program Big Data, Scientific & Engineering Workloads at VMware Corporate Headquarters, Palo Alto, Ca. This program is an extension of the very successful VMWare Expert Programs held all over the world on Oracle and Microsoft SQL Server.
VMware Experts Program Big Data Attendees
Kailash Marthi? EMC, Kris Applegate DELL Tom McCuch HortonWorks, Roman Zajic?Quanta Cloud Technology (QCT),??Tej Luthra Merlin, Jamal Robinson Intel, Dev Reddy, Chris Dagdigian @chris_dag BioTeam, Tony Foster?@wonder_nerd EMC, Motti Beck? @MottiBeck??Mellonox Technologies, Fredrick Van Haren Evaluator Group
To attend the VMware Experts Program Big Data one guest flew in all the way from Taiwan.











AdvanceResource Management -Deep Dive Workshop
& ESXi Scheduler Operation
Mark Achtemichuk Performance Staff Engineer, VMWare
We have optimized the stack for latency in both directions
There isn?t overhead in running virtual machines anymore
TPS is now secure by default
The Protocol war continues, NFS, iSCSI, and FC each are performant, bandwidth required
JUMBO frames Not faster but reduced overhead cost
Right Size, Right Size, Right Size Spend effort on rightsizing workloads for VCPU count and assigned memory
Enable Hyper-Threading – Vsphere understands and uses hyper-threading to its advantage
Don’t use vCPU Hot-Add – As it disables vNUMA and presents the virtual machine with UMA topology
Use vmxnet3 guest network driver ? very efficient and required for maximum performance
Evaluate disabling interrupt coalescing ? default may induce small amounts of latency in favor of throughput, evaluate disabling it as cost today is negligible
It?s a 10Gb+ World ? 1Gb saturation is real, more bandwidth required today. Especially in light of vSAN, Monster BM vMotion
VSphere is tuned to be conservative out of the gate (Default Behavior all flash world, not appropriate out of the gate).
Meeting at the Edge
Ray O’Farrell EVP & CTO VMware

The Internet of Things is an industrial revolution of sorts
How do you leverage the new technologies to change how your company works
Your ability to get at CPU Cycles has dropped in price enormously
New ways of analyzing all this data keep emerging
Challenge of revolutions, people die. So how do we help people make the transitions
Companies that failed to make the transition are likely to be eclipsed by companies that do
All the data is being generated at the edge and needs to be pushed back
Our focus is on Security of infrastructure, management of the infrastructure, performance of the infrastructure
Big Data Reference Architectures for vSphere ? Justin Murray

Customers want to get off costly data platforms
Older data warehouse technology not serving their needs
Provide data access to our customers directly
Exploit new application architecture for developer productivity
S3 is an object storage system rather than a file system, S3 is eventually consistent
The answer is yes we do containers
We suggest you turn hyper-threading on
Extreme?Performance: Fast virtualized Haddop and Spark on All-Flash Disks
Dave?Jaffe

Memory, CPU increasing critical for new technologies like Spark. CPU: Larger core count equally as important as faster clock speed
Use Flash Disks Appropriately
10 GbE crucial, starting to see 25 GbE
Number of Servers determined by size of workload, number of concurrent users
Number of Servers determined by size of workload, number of concurrent users
Use Eager Zero Thick format for virtual disks
Limit number of disks per data node to maximize utilization of each disk ? 4-6 good starting?point
Reserve 5-6% of total server memory for ESXi, use remainder for VMs
dfs.blocksize ? Tradeoff between size and number of tasks ? 256 MB good initial choice for most workloads
dfs.replication ? 3 typical for availability
Powerful new technologies like YARN, Spark and Machine Leasing yield excellent performance on vSphere when tuned properly
Performance Results show virtualized Spark and Hadoop is 10% better than native
Virtualizing HPC: Why and How
Josh Simons, Sr. Director & Chief Technologist High-Performance Computing VMware

Increase IT Agility and Flexibility and decrease time to discovery for researchers, scientist, and engineers
Life science folks want to do their science, they don?t want to do the technology. It?s a means to the end
Benefits of Virtual Machines: Heterogeneity, Multitenant data security, Fault Isolation, Reproducibility, Fault resiliency, Dynamic Load Balance & Performance
Using Compute Accelerators with vSphere increasing common in traditional and emerging HPC (ML/DL, etc.) – GPGPU, FPGA, Intel, Xeon, Phi, Etc
Three approaches Available, – 1. Direct Passthrough (VM Direct Path I/0), 2. PCI SR-IOV for generic device sharing to multiple VMs, 3. NVIDIA GRID for GPGPU(And GPU) sharing and enterprise management




History of VMware Experts Program
The VMware Experts Program began in 2013 as a collaborative effort between the Cloud Platform Business Unit, then referred to as vSphere Product Marketing team and the VMware Global Center of Excellence. The program brings together experts in a variety of applications, database and other technical fields at VMware in Palo Alto (Now globally) to meet with VMware and partner personnel from the highest-level executives to the engineers who architect the platform and write code. A variety of open discussions and exercises facilitate an intense learning environment leading to the expansion of the already broad communities of technical experts that have extensive knowledge of VMware virtualized infrastructure.
VMware Experts Program Videos
Here are some videos from previous versions of the program.
Experts Workshop ? Database Workshop ? Oracle Edition April 2017 ? Palo Alto https://www.youtube.com/watch?v=aVvUNhzZ52Q&feature=youtu.be
VMware Experts ? Database Workshop ? Oracle Edition April 2016 ? Palo Alto
MS SQL Server Workshop Montage Videos
VMware Experts Database Workshop ? SQL Workshop ? Sept 2014 ? Palo Alto
VMware Experts Database Workshop ? SQL Workshop ? April 2015 ? Palo Alto
VMware Experts Database Workshop ? SQL workshop ? Nov 2016 ? Sofia Bulgaria
https://www.youtube.com/watch?v=WR4Q5tQpU2Q
VMware Experts Program Cork Ireland
VMware Experts Program Cork Ireland Montage Video?
Mike
*** All Posts are Mine and Not those of my Employer**
LicenseFortress – Real Time Oracle License Compliance Alerting and Management with a Financial Guarantee.
My Blog: http://michaelcorey.com/
My Personal Twitter Account:?Michael_Corey
Columnist for the Big Data Quarterly. <Click Here to Subscribe Big Data Quarterly>
Buy at VMWarePress!