Go to website Go to website

Citrix guy here. Can we talk about the infrastructure?

  • 28 May 2020
  • 8 replies
  • 501 views
T
Userlevel 2
Badge

I'm a VDI engineer, meaning I build and support Enterprise virtual desktop environments, typically using Citrix technology as the main point of delivery, with others mixed in.

 

Until Onlive and later Sony popularized the idea of cloud gaming to me, the only knowledge I had on the subject was Citrix's old ICA/HDX protocol and later, GPU support. The entire protocol was TCP at the time. They now have UDP support in the protocol, but framerates are still limited at 60 fps. Naturally, it's built for many users using as little bandwidth as possible, not buttery smooth gaming. Shadow, parsec, etc are obviously built for a different purpose.

 

But what does Shadow infra look like?

 

Does it run on KVM? Xen? ESX? 

Is each VM entirely persistent (as they appear to be?)

Hardware wise, is shadow built on hyperconverged?

Is there already some place where I can discuss these things?

 

 

8 replies

Scyne
Badge

I, too, am interested.

J
Badge

Qemu drivers are loaded I noticed that, I will check if I can spot more info 

FrostCone

Yes, I would love to find out :grinning:

Hascrack
Userlevel 3
Badge +4

The hardware itself is split into storage and compute nodes that actually give shadow its umph.

Shadow uses open compute servers with 2 nodes per RU. Each node is split into 2 sets of hardware for VMs. The hypervisor used for this is KVM. The nodes connect to the network with dual 10GbE. Each vm when spun up is given access to 12Gb ram, a quadro P5000 and 4 cores 8 threads of a Xeon (mostly 2667v3s) . When the client disconnects the VM stays running on the server for 90 minutes before hibernating and disconnecting from the node. (in a normal situation excluding the current crysis)

 The storage SANs are all SSD ZFS pools that have 100GbE links to the network. The storage connects via iSCSI targeting. This has been the primary concern when provisioning new customers as they do not want to overburden the pools ith windows installs, updates and new user downloads. Secondary storage is SSD backed HDD storage.

The network itself is a made up of Juniper QFX gear using leaf-spine /super-spine topology Each rack has dual 100GbE up links to the spine.

They leverage in DC transit to accelerate things like windows updates or steam downloads and are working on improvements to their upstream connections. Shadow is looking to improve steam throughout by white listing steam cache servers in DC to decrease firewall load and increase download performance but that is currently in early stages of development.

For out of DC connections they use BGP peering arrangements with tier 1 ISPs to help reduce latency with the end users.

It is only when they go wrong that machines remind you how powerful they are. ~Clive James
T
Userlevel 2
Badge

The hardware itself is split into storage and compute nodes that actually give shadow its umph.

 

 

Thanks for the detailed response! I wondered if hyperconverged would offer better performance than traditional SAN, but naturally offers a different issue with scalability and cost effectiveness.

Great to hear about the cache servers, this is similar to how streaming services deliver content as well.

Hascrack
Userlevel 3
Badge +4


Great to hear about the cache servers, this is similar to how streaming services deliver content as well.

Due to the location of the data centers, high speed interconnects between shadow’s network and CDNs hosting these kind of services make for lower cost options for bandwidth.

It is only when they go wrong that machines remind you how powerful they are. ~Clive James
M
Badge

Hopefully this is still on topic but how does shadow always know the correct disk images to load? Has there or is there any way someone could randomly load the wrong images? How much protection is there to protect against that?

 

Security of this service always concerns me and this is one of the last concerns I have. My memory is telling me I read somewhere of someone getting someone else's shadow disk image but I don’t know if I am miss remembering.

 

Thanks

Hascrack
Userlevel 3
Badge +4

Hopefully this is still on topic but how does shadow always know the correct disk images to load? Has there or is there any way someone could randomly load the wrong images? How much protection is there to protect against that?

 

Security of this service always concerns me and this is one of the last concerns I have. My memory is telling me I read somewhere of someone getting someone else's shadow disk image but I don’t know if I am miss remembering.

 

Thanks

Example VHDX, note the file owner (Hyper-V)

Like Most Hypervisors, KVM assigns a Unique identifier to every VM and the virtual disk is tied to this. without the virtual machine having permission to access the disk, it will be unable to load. Furthermore with iSCSI there is added layers of security on top of what is provided by the hypervisor / host os. A good article on iSCSI security can be found here https://www.inap.com/blog/block-storage-security-101-an-introduction-to-iscsi-san-deployments-and-security/ . If you do have security concerns that these things do not address I recommend getting additional storage and encrypting it with bitlocker for an added layer of security.

It is only when they go wrong that machines remind you how powerful they are. ~Clive James

Reply


Terms & ConditionsCookie settings