fidzu : pda

Posted by Benjamin Baxter, Developer Advocate and Bacon Connoisseur

All TVs have the same problem with keyboard input: It is very cumbersome to hunt and peck for each letter using a D-pad with a remote. And if you make a mistake, trying to correct it compounds the problem.

APIs like Smart Lock and Autofill, can ease user's frustrations, but for certain types of input, like login, you need to collect complex input that is difficult for users using the on-screen keyboard.

With the Nearby Connections API, you can use a second screen to gather input from the user with less friction.

How Nearby Connections works

From the documentation:

"Nearby Connections is an offline peer-to-peer socket model for communication based on advertising and discovering devices in proximity.

Usage of the API falls into two phases: pre-connection, and post-connection.

In the pre-connection phase, Advertisers advertise themselves, while Discoverers discover nearby Advertisers and send connection requests. A connection request from a Discoverer to an Advertiser initiates a symmetric authentication flow that results in both sides independently accepting (or rejecting) the connection request.

After a connection request is accepted by both sides, the connection is established and the devices enter the post-connection phase, during which both sides can exchange data."

In most cases the TV is the advertiser and the phone is the discoverer. In the example below, the assumed second device is a phone. The API and patterns described in this article are not limited to a phone. For example, a tablet could also be the second screen device.

Login Example

There are many times when keyboard input is required. Authenticating users and collecting billing information (like zip codes and name on card) are common cases. This example handles a login flow that uses a second screen to see how Nearby Connections can help reduce friction.

1. The user opens your app on her TV and needs to login. You can show a screen of options similar to the setup flow for a new TV.

2. After the user chooses to login with their phone, the TV should start advertising and send the user to the associated login app on their phone, which should start discovering.

There are a variety of solutions to open the app on the phone. As an example, Android TV's setup flow has the user open the corresponding app on their mobile device. Initiating the hand-off is a more a UX concern than a technology concern.

3. The phone app should display the advertising TV and prompt the user to initiate the connection. After the (encrypted -- see Security Considerations below for more on this) connection is established the TV can stop advertising and the phone can stop discovering.

"Advertising/Discovery using Nearby Connections for hours on end can affect a device's battery. While this is not usually an issue for a plugged-in TV, it can be for mobile devices, so be conscious about stopping advertising and discovery once they're no longer needed."

4. Next, the phone can start collecting the user's input. Once the user enters their login information, the phone should send it to the TV in a BYTES payload over the secure connection.

5. When the TV receives the message it should send an ACK (using a BYTES payload) back to the phone to confirm delivery.

6. When the phone receives the ACK, it can safely close the connection.

The following diagram summarizes the sequence of events:

UX considerations

Nearby Connections needs location permissions to be able to discover nearby devices. Be transparent with your users. Tell them why they need to grant the location permission on their phone.

Since the TV is advertising, it does not need location permissions.

Start advertising: The TV code

After the user chooses to login on the phone, the TV should start advertising. This is a very simple process with the Nearby API.

override fun onGuidedActionClicked(action: GuidedAction?) {
    super.onGuidedActionClicked(action)
    if( action == loginAction ) {
        // Update the UI so the user knows to check their phone
        navigationFlowCallback.navigateToConnectionDialog()
        doStartAdvertising(requireContext()) { payload ->
            handlePayload(payload)
        }
    }
}

When the user clicks a button, update the UI to tell them to look at their phone to continue. Be sure to offer a way to cancel the remote login and try manually with the cumbersome onscreen keyboard.

This example uses a GuidedStepFragment but the same UX pattern applies to whatever design you choose.

Advertising is straightforward. You need to supply a name, a service id (typically the package name), and a `ConnectionLifeCycleCallback`.

You also need to choose a strategy that both the TV and the phone use. Since it is possible that the users has multiple TVs (living room, bedroom, etc) the best strategy to use is P2P_CLUSTER.

Then start advertising. The onSuccessListener and onFailureListener tell you whether or not the device was able to start advertising, they do not indicate a device has been discovered.

fun doStartAdvertising(context: Context) {
    Nearby.getConnectionsClient(context).startAdvertising(
        context.getString(R.string.tv_name),
        context.packageName,
        connectionLifecycleCallback,
        AdvertisingOptions.Builder().setStrategy(Strategy.P2P_CLUSTER).build()
    )
    .addOnSuccessListener {
        Log.d(LoginStepFragment.TAG, "We are advertising!")
    }
    .addOnFailureListener {
        Log.d(LoginStepFragment.TAG, "We cannot start advertising.")
        Toast.makeText(
            context, "We cannot start advertising.", Toast.LENGTH_LONG)
                .show()
    }
}

The real magic happens in the `connectionLifecycleCallback` that is triggered when devices start to initiate a connection. The TV should accept the handshake from the phone (after performing the necessary authentication -- see Security Considerations below for more) and supply a payload listener.

val connectionLifecycleCallback = object : ConnectionLifecycleCallback() {

    override fun onConnectionInitiated(
            endpointId: String, 
            connectionInfo: ConnectionInfo
    ) {
        Log.d(TAG, "Connection initialized to endpoint: $endpointId")
        // Make sure to authenticate using `connectionInfo.authenticationToken` 
        // before accepting
        Nearby.getConnectionsClient(context)
            .acceptConnection(endpointId, payloadCallback)
    }

    override fun onConnectionResult(
        endpointId: String, 
        connectionResolution: ConnectionResolution
    ) {
        Log.d(TAG, "Received result from connection: ${connectionResolution.status.statusCode}")
        doStopAdvertising()
        when (connectionResolution.status.statusCode) {
            ConnectionsStatusCodes.STATUS_OK -> {
                Log.d(TAG, "Connected to endpoint: $endpointId")
                otherDeviceEndpointId = endpointId
            }
            else -> {
                otherDeviceEndpointId = null
            }
        }
    }

    override fun onDisconnected(endpointId: String) {
        Log.d(TAG, "Disconnected from endpoint: $endpointId")
        otherDeviceEndpointId = null
    }
}

The payloadCallback listens for the phone to send the login information needed. After receiving the login information, the connection is no longer needed. We go into more detail later in the Ending the Conversation section.

Discovering the big screen: The phone code

Nearby Connections does not require the user's consent. However, the location permission must be granted in order for discovery with Nearby Connections to work its magic. (It uses BLE scanning under the covers.)

After opening the app on the phone, start by prompting the user for location permission if not already granted on devices running Marshmallow and higher.

Once the permission is granted, start discovering, confirm the connection, collect the credentials, and send a message to the TV app.

Discovering is as simple as advertising. You need a service id (typically the package name -- this should be the same on the Discoverer and Advertiser for them to see each other), a name, and a `EndpointDiscoveryCallback`. Similar to the TV code, the flow is triggered by callbacks based on the connection status.

Nearby.getConnectionsClient(context).startDiscovery(
        context.packageName,
        mobileEndpointDiscoveryCallback,
        DiscoveryOptions.Builder().setStrategy(Strategy.P2P_CLUSTER).build()
        )
        .addOnSuccessListener {
            // We're discovering!
            Log.d(TAG, "We are discovering!")
        }
         .addOnFailureListener {
            // We were unable to start discovering.
            Log.d(TAG, "We cannot start discovering!")
        }

The Discoverer's listeners are similar to the Advertiser's success and failure listeners; they signal if the request to start discovery was successful or not.

Once you discover an advertiser, the `EndpointDiscoveryCallback` is triggered. You need to keep track of the other endpoint to know who to send the payload, e.g.: the user's credentials, to later.

val mobileEndpointDiscoveryCallback = object : EndpointDiscoveryCallback() {
    override fun onEndpointFound(
        endpointId: String, 
        discoveredEndpointInfo: DiscoveredEndpointInfo
    ) {
        // An endpoint was found!
        Log.d(TAG, "An endpoint was found, ${discoveredEndpointInfo.endpointName}")
        Nearby.getConnectionsClient(context)
            .requestConnection(
                context.getString(R.string.phone_name), 
                endpointId, 
                connectionLifecycleCallback)
    }

    override fun onEndpointLost(endpointId: String) {
        // A previously discovered endpoint has gone away.
        Log.d(TAG, "An endpoint was lost, $endpointId")
    }
}

One of the devices must initiate the connection. Since the Discoverer has a callback for endpoint discovery, it makes sense for the phone to request the connection to the TV.

The phone asks for a connection supplying a `connectionLifecycleCallback` which is symmetric to the callback in the TV code.

val connectionLifecycleCallback = object : ConnectionLifecycleCallback() {
    override fun onConnectionInitiated(
        endpointId: String,
        connectionInfo: ConnectionInfo
    ) {
        Log.d(TAG, "Connection initialized to endpoint: $endpointId")
        // Make sure to authenticate using `connectionInfo.authenticationToken` before accepting
        Nearby.getConnectionsClient(context)
                .acceptConnection(endpointId, payloadCallback)
    }

    override fun onConnectionResult(
        endpointId: String,
        connectionResolution: ConnectionResolution
    ) {
        Log.d(TAG, "Connection result from endpoint: $endpointId")
        when (connectionResolution.status.statusCode) {
            ConnectionsStatusCodes.STATUS_OK -> {
                Log.d(TAG, "Connected to endpoint: $endpointId")
                otherDeviceEndpointId = endpointId
                waitingIndicator.visibility = View.GONE
                emailInput.editText?.isEnabled = true
                passwordInput.editText?.isEnabled = true

                Nearby.getConnectionsClient(this).stopDiscovery()
            }
            else -> {
                otherDeviceEndpointId = null
            }
        }
    }

    override fun onDisconnected(endpointId: String) {
        Log.d(TAG, "Disconnected from endpoint: $endpointId")
        otherDeviceEndpointId = null
    }
}

Once the connection is established, stop discovery to avoid keeping this battery-intensive operation running longer than needed. The example stops discovery after the connection is established, but it is possible for a user to leave the activity before that happens. Be sure to stop the discovery/advertising in onStop() on both the TV and phone.

override fun onStop() {
    super.onStop()
    Nearby.getConnectionsClient(this).stopDiscovery()
}




Just like a TV app, when you accept the connection you supply a payload callback. The callback listens for messages from the TV app such as the ACK described above to clean up the connection.

After the devices are connected, the user can use the keyboard and send their authentication information to the TV by calling `sendPayload()`.

fun sendCreditials() {

    val email = emailInput.editText?.text.toString()
    val password = passwordInput.editText?.text.toString()

    val creds = "$email:$password"
    val payload = Payload.fromBytes(creds.toByteArray())
    Log.d(TAG, "sending payload: $creds")
    if (otherDeviceEndpointId != null) {
        Nearby.getConnectionsClient(this)
                .sendPayload(otherDeviceEndpointId, payload)
    }
}

Ending the conversation

After the phone sends the payload to the TV (and the login is successful), there is no reason for the devices to remain connected. The TV can initiate the disconnection with a simple shutdown protocol.

The TV should send an ACK to the phone after it receives the credential payload.

val payloadCallback = object : PayloadCallback() {
    override fun onPayloadReceived(endpointId: String, payload: Payload) {
        if (payload.type == Payload.Type.BYTES) {
            payload.asBytes()?.let {
                val body = String(it)
                Log.d(TAG, "A payload was received: $body")
                // Validate that this payload contains the login credentials, and process them.

                val ack = Payload.fromBytes(ACK_PAYLOAD.toByteArray())
                Nearby.getConnectionsClient(context).sendPayload(endpointId, ack)
            }
        }
    }

    override fun onPayloadTransferUpdate(
        endpointId: String,
        update: PayloadTransferUpdate
    ) {    }
}

The phone should have a `PayloadCallback` that initiates a disconnection in response to the ACK. This is also a good time to reset the UI to show an authenticated state.

private val payloadCallback = object : PayloadCallback() {
    override fun onPayloadReceived(endpointId: String, payload: Payload) {
        if (payload.type == Payload.Type.BYTES) {
            payload.asBytes()?.let {
                val body = String(it)
                Log.d(TAG, "A payload was received: $body")

                if (body == ACK_PAYLOAD) {
                    waitingIndicator.visibility = View.VISIBLE
                    waitingIndicator.text = getString(R.string.login_successful)
                    emailInput.editText?.isEnabled = false
                    passwordInput.editText?.isEnabled = false
                    loginButton.isEnabled = false

                    Nearby.getConnectionsClient(this@MainActivity)
                        .disconnectFromEndpoint(endpointId)
                }
            }
        }
    }

    override fun onPayloadTransferUpdate(
        endpointId: String,
        update: PayloadTransferUpdate
    ) {    }
}

Security considerations

For security (especially since we're sending over sensitive information like login credentials), it's strongly recommended that you authenticate the connection by showing a code and having the user confirm that the two devices being connected are the intended ones -- without this, the connection established by Nearby Connection is encrypted but not authenticated, and that's susceptible to Man-In-The-Middle attacks. The documentation goes into greater detail on how to authenticate a connection.

Does your app offer a second screen experience?

There are many times when a user needs to supply input to a TV app. The Nearby API provides a way to offload the hardships of an onscreen-dpad-driven keyboard to an easy and familiar phone keyboard.

What use cases do you have where a second screen would simplify your user's life? Leave a comment or send me (@benjamintravels) or Varun (@varunkapoor, Team Lead for Nearby Connections) a tweet to continue the discussion.

17 Aug 2018 10:33pm GMT

Posted by Seang Chau, Vice President, Engineering

According to the World Health Organization¹, around 466 million people worldwide have disabling hearing loss. This number is expected to increase to 900 million people by the year 2050. Google is working with GN Hearing to create a new open specification for hearing aid streaming support on future versions of Android. Users with hearing loss will be able to connect, pair, and monitor their hearing aids so they can hear their phones loudly and clearly.

Hearing aid users expect a high quality, low latency experience with minimal impact on phone and hearing aid battery life. We've published a new hearing aid spec for Android smartphones: Audio Streaming for Hearing Aids (ASHA) on Bluetooth Low Energy Connection-Oriented Channels. ASHA is designed to have a minimal impact on battery life with low-latency while maintaining a high quality audio experience for users who rely on hearing aids. We look forward to continually evolving the spec to even better meet the needs of our users.

The spec details the pairing and connectivity, network topology, system architecture, and system requirements for implementing hearing aids using low energy connection-oriented channels. Any hearing aid manufacturer can now build native hearing aid support for Android.

The protocol specification is available here.

16 Aug 2018 1:00pm GMT

Posted by Hoi Lam, Lead Developer Advocate, Wear OS by Google

Today we are announcing a new initiative to improve Wear app quality and their presentation in the Google Play Store. The Wear app review process, which has been in place since the launch of Android Wear 2.0, is currently optional. It will become mandatory for apps to be listed on the Wear OS by Google version of the Google Play Store from the following dates:

• New Wear apps: 1 October 2018
• Existing Wear apps: 4 March 2019.

The review process for mobile apps remains unchanged, and is independent of the Wear app review. Mobile app updates will not be blocked if they fail the Wear app review.

We hope this lightweight app review process will improve the quality of Wear app experiences across the wide range of devices available to your users. In addition, since screenshots are required for the Wear app review, this will improve the discovery and presentation of your Wear apps in the Google Play Store.

See a comprehensive list of review criteria here. The following are common issues we see during Wear app reviews:

• Support for different screen types - Wear OS by Google is available in both round and square screens, and some round devices also have a chin. Developers are advised to test on all screen types. If a physical device is unavailable, please use the Wear OS by Google emulator.
• Wear OS by Google app screenshot - To pass the review, the app needs to have at least one Wear OS app screenshot. To keep pre-release Wear apps private, the Google Play Store will not show the Wear screenshots unless the Wear App is in production or open testing. Currently, the Google Play Store only supports uploading one set of screenshots across all production and test versions. For existing Wear apps, we recommend developers keeping their production Wear app screenshots unchanged when uploading new open test or closed test Wear apps.

Opting out of app review for early prototypes

We understand that some developers need to experiment with their Wear apps in the early stages of app development, and a Wear app review at this stage might not be appropriate. In this case, developers have two options:

• Manually deploy Wear APKs to their users, or
• Leverage internal testing features via the Google Play console, which enables developers to test with up to 100 internal testing accounts.

Please note that the open test and closed test channels will be subject to Wear app review to help front-load the quality assurance process and to avoid leaving reviews to the last minute.

Thank you for your continuing support of Wear OS by Google.

15 Aug 2018 6:12pm GMT