VMware Experts Program Big Data

VMware Experts Program Big Data

This week VMware is holding the VMware Experts Program Big Data, Scientific & Engineering Workloads at VMware Corporate Headquarters, Palo Alto, Ca. This program is an extension of the very successful VMWare Expert Programs held all over the world on Oracle and Microsoft SQL Server.

VMware Experts Program Big Data

VMware Experts Program Big Data Attendees

Kailash Marthi  EMC, Kris Applegate DELL Tom McCuch HortonWorks, Roman Zajic Quanta Cloud Technology (QCT),  Tej Luthra Merlin, Jamal Robinson Intel, Dev Reddy, Chris Dagdigian @chris_dag BioTeam, Tony Foster @wonder_nerd EMC, Motti Beck  @MottiBeck  Mellonox Technologies, Fredrick Van Haren Evaluator Group

To attend the VMware Experts Program Big Data one guest flew in all the way from Taiwan.

Tej Luthra Merlin
Tej Luthra Merlin
Tom McCuch HortonWorks
Tom McCuch HortonWorks
Roman Zajic
Roman Zajic QCT
Jamal Robinson Intel
Jamal Robinson Intel

Kailash Marthi EMC
Kris Applegate DELL
Kris Applegate DELL
Tony Foster EMC
Tony Foster EMC
Fredrick Van Haren Evaluator Group
Fredrick Van Haren Evaluator Group

 

 

Dev Reddy
Dev Reddy
Chris Dagdigian BioTeam
Chris Dagdigian BioTeam
Motti Beck Mellanox
Motti Beck Mellanox

 

 

 

AdvanceResource Management -Deep Dive Workshop

& ESXi Scheduler Operation

Mark Achtemichuk Performance Staff Engineer, VMWare

Mark Achtemichuk

 

 

 

 

 

 

We have optimized the stack for latency in both directions

There isn’t overhead in running virtual machines anymore

TPS is now secure by default

The Protocol war continues, NFS, iSCSI, and FC each are performant, bandwidth required

JUMBO frames Not faster but reduced overhead cost

Right Size, Right Size, Right Size Spend effort on rightsizing workloads for VCPU count and assigned memory

Enable Hyper-Threading – Vsphere understands and uses hyper-threading to its advantage

Don’t use vCPU Hot-Add – As it disables vNUMA and presents the virtual machine with UMA topology

Use vmxnet3 guest network driver – very efficient and required for maximum performance

Evaluate disabling interrupt coalescing – default may induce small amounts of latency in favor of throughput, evaluate disabling it as cost today is negligible

It’s a 10Gb+ World – 1Gb saturation is real, more bandwidth required today. Especially in light of vSAN, Monster BM vMotion

VSphere is tuned to be conservative out of the gate (Default Behavior all flash world, not appropriate out of the gate).

Meeting at the Edge

Ray O’Farrell EVP & CTO VMware

Ray O'Farrell EVP & CTO VMware
Ray O’Farrell EVP & CTO VMware

 

 

 

 

 

 

 

 

The Internet of Things is an industrial revolution of sorts

How do you leverage the new technologies to change how your company works

Your ability to get at CPU Cycles has dropped in price enormously

New ways of analyzing all this data keep emerging

Challenge of revolutions, people die. So how do we help people make the transitions

Companies that failed to make the transition are likely to be eclipsed by companies that do

All the data is being generated at the edge and needs to be pushed back

Our focus is on Security of infrastructure, management of the infrastructure, performance of the infrastructure

Big Data Reference Architectures for vSphere – Justin Murray

Justin Murray VMware
Justin Murray VMware

 

 

 

 

 

 

 

 

Customers want to get off costly data platforms

Older data warehouse technology not serving their needs

Provide data access to our customers directly

Exploit new application architecture for developer productivity

S3 is an object storage system rather than a file system, S3 is eventually consistent

The answer is yes we do containers

We suggest you turn hyper-threading on

Extreme Performance: Fast virtualized Haddop and Spark on All-Flash Disks

Dave Jaffe

Dave Jaffe Vmware
Dave Jaffe VMware

 

 

 

 

 

 

 

 

Memory, CPU increasing critical for new technologies like Spark. CPU: Larger core count equally as important as faster clock speed

Use Flash Disks Appropriately

10 GbE crucial, starting to see 25 GbE

Number of Servers determined by size of workload, number of concurrent users

Number of Servers determined by size of workload, number of concurrent users

Use Eager Zero Thick format for virtual disks

Limit number of disks per data node to maximize utilization of each disk – 4-6 good starting point

Reserve 5-6% of total server memory for ESXi, use remainder for VMs

dfs.blocksize – Tradeoff between size and number of tasks – 256 MB good initial choice for most workloads

dfs.replication – 3 typical for availability

Powerful new technologies like YARN, Spark and Machine Leasing yield excellent performance on vSphere when tuned properly

Performance Results show virtualized Spark and Hadoop is 10% better than native

 

Virtualizing HPC: Why and How

Josh Simons, Sr. Director & Chief Technologist High-Performance Computing VMware

Josh Simons VMware
Josh Simons Sr. Director & Chief Technologist, High Performance Computing, VMware

 

 

 

 

 

 

 

 

 

Increase IT Agility and Flexibility and decrease time to discovery for researchers, scientist, and engineers

Life science folks want to do their science, they don’t want to do the technology. It’s a means to the end

Benefits of Virtual Machines: Heterogeneity, Multitenant data security, Fault Isolation, Reproducibility, Fault resiliency, Dynamic Load Balance & Performance

Using Compute Accelerators with vSphere increasing common in traditional and emerging HPC (ML/DL, etc.) – GPGPU, FPGA, Intel, Xeon, Phi, Etc

Three approaches Available, – 1. Direct Passthrough (VM Direct Path I/0), 2. PCI SR-IOV for generic device sharing to multiple VMs, 3. NVIDIA GRID for GPGPU(And GPU) sharing and enterprise management

Don Sullivan VMware
Don Sullivan VMware
Dean Bolton LicenseFortress
Dean Bolton, LicenseFortress

 

 

 

 

 

 

 

 

Charu Chaubal VMware
Charu Chaubal VMware
Gina Rosenthal VMware
Gina Rosenthal VMware

 

 

 

 

 

 

History of VMware Experts Program

The VMware Experts Program began in 2013 as a collaborative effort between the Cloud Platform Business Unit, then referred to as vSphere Product Marketing team and the VMware Global Center of Excellence. The program brings together experts in a variety of applications, database and other technical fields at VMware in Palo Alto (Now globally) to meet with VMware and partner personnel from the highest-level executives to the engineers who architect the platform and write code. A variety of open discussions and exercises facilitate an intense learning environment leading to the expansion of the already broad communities of technical experts that have extensive knowledge of VMware virtualized infrastructure.

VMware Experts Program Videos

Here are some videos from previous versions of the program.

Experts Workshop – Database Workshop – Oracle Edition April 2017 – Palo Alto https://www.youtube.com/watch?v=aVvUNhzZ52Q&feature=youtu.be

VMware Experts – Database Workshop – Oracle Edition April 2016 – Palo Alto

https://youtu.be/7BzIQbJ5g28

MS SQL Server Workshop Montage Videos

VMware Experts Database Workshop – SQL Workshop – Sept 2014 – Palo Alto

http://youtu.be/Tydd1LJJ4wM

http://michaelcorey.ntirety.com/blog/tabid/51101/bid/108324/Elite-SQL-Server-database-virtualization-Workshop-Day-1-3.aspx

VMware Experts Database Workshop – SQL Workshop – April 2015 – Palo Alto

https://youtu.be/gyDdLm5ky-8

http://michaelcorey.ntirety.com/blog/vmware-database-workshop-microsoft-sql-server-with-tintri-day-1-3

VMware Experts Database Workshop – SQL workshop – Nov 2016 – Sofia Bulgaria

https://www.youtube.com/watch?v=WR4Q5tQpU2Q

VMware Experts Program Cork Ireland

VMware Experts Program Cork Ireland Montage Video 

Mike

*** All Posts are Mine and Not those of my Employer**

LicenseFortress – Real Time Oracle License Compliance Alerting and Management with a Financial Guarantee.

My Blog: http://michaelcorey.com/

My Personal Twitter Account: Michael_Corey

Columnist for the Big Data Quarterly. <Click Here to Subscribe Big Data Quarterly>

LicenseFortress

 

 

 

Buy at VMWarePress!

Virtualizing SQL Server G


Vmware vExpertOracle Ace

 

 

 

 

 

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.