Introducing a Streaming Video Service That Syncs With All Your Lights

Hey there,

Kevin from Light DJ here. It’s been a while since my last post but that’s because I’ve been plugging away at something brand new and I’m finally ready to share it with all of you, but some back story first.

Light DJ blew up a bit during COVID when lots of people where bored at home and streaming concerts were on the rise. I watched a few of those while controlling my lights by hand but couldn't help but wish there were a better way to sync them to the video. After 8 years of developing light apps I understood many of the technical challenges in making that happen and have seen various attempts at a solution:

• You can use a Hue Sync box, but then you're limited to HDMI, only Hue products, and only screen mirroring effects.

• You can use a Nanoleaf 4D camera, but then you're limited to Nanoleaf and screen mirroring effects.

• You can use the Hue Sync TV app, but then you're limited to Hue on certain TVs.

• SYFY tried something called SYFY Sync. It was a bit ahead of its time but only worked on certain shows with Hue and has since been decommissioned.

• LIFX users are left out in the cold.

So I sought to create a system that could work with any brand of light and that didn't need any extra hardware. TL;DR after 10 months of research and development I'm finally ready share my creation with the world.

I call it LYTVOX (pronounced "light-vocks") and it's an entire light show authoring pipeline that allows content producers to easily create ambient lighting experiences that sync with their video content, and enables streaming media providers to deliver those experiences to users.

You can see a demo here:

Some of the benefits of our solution include:

• Fully customizable 4K surround lighting video presentations streamed directly to users.

• Creators have full control of the light show authoring process and are not just limited to screen mirroring; they have the ability colorize the entire 3D space frame-by-frame.

• It can support any combination of lighting hardware & any number of lights.

• Adapts to your room based on light placement for the full surround lighting experience.

• tvOS and Google TV apps for accessing content directly from TVs and streaming devices like Apple TV 4K and Chromecast.

• iOS and Android mobile apps for broad compatibility, including Airplay and Google Cast support.

• Light hardware agnostic - we can support any light that can be controlled on the local network at a certain rate (currently Hue, LIFX, and Nanoleaf).

• No additional cameras or boxes needed; runs on devices most users already own.

So how can you help us?

We're currently seeking Early Access beta testers who would like to try our client app and give us feedback. We have over 5 hours of content for you to enjoy and about 30 different types of videos available, including music videos, concerts, anime, ambiences, and more. Beta access is completely free. To access the beta you must have an iOS/tvOS or Android device and supported smart lighting:

• Hue (v2 bridge required)

• Nanoleaf (all panel types)

• LIFX (all devices)

If this sounds like something you're interested in, head over to our website and submit your email address for early access: www.lytvox.com

If you have any thoughts or ideas about what you'd like to see with this service, drop a comment.

What about Light DJ?

Don’t worry - Light DJ is here to stay! And as a bonus, a lot of the work I did preparing LYTVOX for production will flow down the all Light DJ apps for iOS and Android. Some notable improvements include:

• All entertainment effects work with LIFX lights

• Hue Gradient Strip / Entertainment v2 support

• All light brands can be now be mixed freely using entertainment effects

  • (Legacy Hue lights are still limited)

• Nanoleaf panels can be controlled as a positionable light for spatial effects

and more, coming to you via an app update in the not so distant future so stay tuned!

Creating stunning visuals using Hue Entertainment

As many of you know, I updated the Light DJ apps earlier this year to support the new Hue Entertainment API. It took just over a month to recreate all of Light DJs 30+ effects in the new format for both iOS and Android. Today I'd like to delve a bit deeper into what it takes to write effects for Hue Entertainment and how you can integrate this new technology into your projects.

The starting place for any Hue developer is the Hue Developer Portal at www.developers.meethue.com. Here you'll find all of the documentation on Hue APIs and guidance for application design. Once you create an account, you'll also need to accept the terms & conditions if you plan on using the Hue Entertainment Development Kit (recommended). The EDK handles bridge discovery & pairing, effect creation & rendering, and streaming to the bridge. If this all sounds complicated, it's not. Hue developed a Quick Start app that incorporates everything you need to start creating effects. (You'll get access to this once you accept the T&C).

Once you start the Quick Start app and pair it with your bridge you'll see two buttons for controlling the lights. The Randomize Lights button uses the legacy Hue API to set the lights to random colors. A typical legacy light command involves a target color and duration. For example, if a light is currently set to blue, you could send the bridge a command to set it to red over 3 seconds and the light will slowly fade from blue to red. If you don't have a need for high-performance or spatial effects this may be enough for your project since it's a bit easier to program for - just be sure you're not exceeding the 10 commands/sec limit on light changes (1 command/sec on groups) or the bridge may start rejecting your commands.

For more advanced effects you’ll need to use the EDK; the Explode button shows an example of this. This process is a bit more involved so I'll go further in detail here. Upon pressing Explode the app will first check to see if you've created an Entertainment Area. If not, it will create one automatically (though I recommend setting one up right from the Hue app so that you can position your lights correctly). Next, the app sets the bridge into streaming mode. This is necessary for the bridge to be able to listen to the UDP-type messages coming from the app. When in streaming mode, the bridge will ignore legacy Hue commands so you'll need to disable it when you're finished if you plan on controlling your lights through other apps. With streaming mode enabled, the app will call a function in the EDK to create an effect based on the parameters that you set. In this case, we're creating an Explosion Effect with an intense burst of color in one location that radiates outward. The effect object is added to the mixer, which renders the state of the lights for the entire Entertainment Area.

Unlike the legacy Hue commands which offer the ability to set individual light states, the EDK always controls the entire Entertainment Area and it will continually stream updates to the lights based on the effects that you add to the mixer. As the name implies, multiple effects can be mixed together with a combination of layers and color transparency.

Aside from the Explosion effect, there are other types of effects that you can create with Hue Entertainment, such as:

• Area Effect - sets all of the lights in sub-area(s) of the Entertainment Area to a color or animation.

• Multi-Channel Effect - sort of like 5.1 audio, you can define a separate channel for different animations. Light DJ typically uses these over Area Effects because I can’t always guarantee that users will have lights in every area. This effect distributes the channels evenly regardless of light locations in the Entertainment Area or how many lights the user has.

• Light Source Effect - created a radial color animation from a specific location in the room. (An Explode Effect is a type of Light Source Effect.)

• Light Iterator Effect - cycles through each of the lights in a room with a certain color or animation.

You can find more details on these effect types at http://developers.meethue.com/documentation/hue-edk-effect-creation including sample code and details on how to define animations.

For advanced users, there's also the ability to create your own fully custom effects, rendered by the EDK, by implementing the Color Animation Effect Delegate. By overriding the color(...), render(), and renderUpdate() functions you can set the color of each light every time the EDK requests a render (around 30 times/sec). If you plan on creating long-running effect sequences, there’s also a Sequence Effect and timeline functions that you can use to program and chain multiple effects over a longer duration.

Light DJ takes advantage of all of these effect types in various places throughout the app. For example, the Strobe Cycle effect uses a Multi-Channel effect where each channel is defined by the four corners of the room, with 1 channel set to the selected color, and 3 channels set to off (black), switching the colored channel each beat. The Drip effect uses an Area Effect (area == .all), displaying a looping fade-off animation over the whole area. The Matrix and Fireworks effects use multiple layered Explode effects. I had to write custom effects for Swirl, Groove Wave and others due to the need to match the speed of the effect to the tempo, but these could have also been made using Sequences. As you use the Light DJ app, see if you can identify the EDK effect(s) I used to make each of the effects.

As you can see the Hue EDK is a powerful tool for creating effects. Before this became available developers had the complex task of managing individual light states on systems with ’n’ number of lights (with significant performance limitations). Now the effects ‘just work’ regardless of the each users' setup. Since integrating these new effects, users’ time-in-app has increased by over 30%. As Philips Hue continues to dominate the smart home market, expect to see more Hue Entertainment integrations with video games and other media. Razer Chroma already works with Hue Entertainment on games like Overwatch, syncing lighting effects with in-game actions.

I hope this helps new developers get up to speed quickly with the new API. If you’re just starting out I recommend creating a few effects right in the Quick Start app for easy development then port them into your own projects. The Quick Start app is available for iOS and Android and EDK library is written in C++ so it’s compatible with most coding projects. If you have any questions about the Hue EDK or creating effects feel free to email me anytime at kevin@nrthrnlights.com.

Cheers and happy coding,

Kevin