User Feedback and Offer to Contribute

javi

Glad to see you almost finish the module.

The missing part is something that isn't documented yet.

You just need to add the following prop to the 'scroll-area' context object. Basically it tells to the tool when to take control.

  switchOnSelectElement: (el) => el.tagName === 'DIV',

javi

I forgot to tell that as part of the action arguments you can find the selected element

i.e.

{
        name: 'down',
        action: ({selectedElement}) => {
          //scroll down the selectedElement
        },
        group: 'Scroll Direction',
        help: 'Scrolls a selected element down',
        switchToContext: 'root'
}

freelyassociate

javi Thank you! Just noticing one issue now when the label picker is highlighting all the divs on the webpage. As there can be quite a lot of divs on a webpage, this can make the number selection process quite messy. Furthermore, not everything that can be scrolled is a div (sometimes it's an unordered list for example).

Instead of making the label picker look for specific elements like a div or ul, is it possible to make the label picker only highlight scrollable elements on a page? That is, elements where the current scrollHeight is greater than the clientHeight? I think this will make the UI and interactions a lot cleaner, but I am not sure if it's possible. What do you think? Thanks again!

javi

I think it makes sense, and you are also correct, at this moment there is no way to select elements using such level of a custom selector it just accepts css queries. i.e. div.some-class

In the meantime, while I think how to support custom element selectors-

What do you think if before returning the object that asks for the pickLabel command, you walk over the elements and add a CSS class or another kind of attribute to those elements that can be scrolled? If you do that then you can change the current label from div to div.scrollable

freelyassociate

javi Good idea, the workaround works! This is what I have so far: https://github.com/leekevinyg/handsfree-for-web-scroll-area-module/blob/master/src/index.js

I am noticing a new issue where:

1) User says "scroll area" and scrollable areas on the page are highlighted
2) User selects a scrollable ul element to scroll using the element picker
3) Instead of the scroll-area context taking control, the text-actions context takes control.

This looks like a conflict in the switchOnSelectElement logic. Currently I have the following for the scroll-area context:

{
      context: 'scroll-area',
      switchOnSelectElement: (el) => el.classList.contains('hands-free-scrollable'),
}

hands-free-scrollable is the className that I am adding to scrollable elements on the page.

Can you advise on why the text-actions context is taking precedent over scroll-area? Thank you!

javi

You're almost there!

I just published a new version of the extension including the following changes:

If the selected element has the class name 'hands-free-scrollable' don't activate the text actions
Share a new parameter called tools when executing an action. The tool parameter is an object that shares relevant helpers or libraries such as: lodash, jquery, moment, domElementTypes and scroll.

I think you will find interesting two of these libs.

scroll: it implements four actions: up(el), down(el), top(el) and bottom(el). It uses jquery to animate the scrolling, there is also a second optional parameter you can use that is distance if you omit it a default distance is calculated based on the height of the element.
domElementTypes: You can see the docs here. https://github.com/sljavi/dom-element-types I think, you might want to use isVisible(el) and also it will be great if at some point we add a new method called isScrollable

Sometimes it takes time to Google Store to publish a new version of the extension, make sure of having the version 3.3.4 of the tool before looking for these changes

Have a great hands-free day 🙂

freelyassociate

javi Thanks Javi! The text actions are not activated now in the new version 3.3.4, and I am able to execute the up/down scroll actions. I am trying to integrate the scroll tool into the action at the moment and have tried passing both tool and tools as action parameter in the following way shown below, but it does not seem to be working. Can you give me an example of how the tools parameter is intended to be passed? Thank you!

{
        name: 'down',
        action: ({ selectedElement, tools }) => {
          tools.scroll.down(selectedElement);
          //selectedElement.scrollTop += 100
        },
        group: 'Scroll Direction',
        help: 'Scrolls a selected element down'
      }

Note the above snippet is from this line inside the scroll-area context: https://github.com/leekevinyg/handsfree-for-web-scroll-area-module/blob/cc54c15294fd87a2cbc62c1d5a9d245e96e3adf9/src/index.js#L48

javi

I just checked the latest version of the module and I can confirm that the scroll object is inside the tools parameter as your code shown.

I have found another issue that is preventing the scrolling. The selectedElement parameter has to be inside a jquery wrapper in order to play correctly with the scroll helpers. I'll change the implementation of it now to avoid requiring the wrapper and I'll release a new version of the extension but it will take a while until it gets published. In the meantime I recommend you change the action to

action: ({ selectedElement, tools }) => {
  tools.scroll.down(tools.jQuery(selectedElement));
},

I recommend you add a property called name to the object that defines the context 'scroll-area'. It will replace the Global label that is next to the mic picture after selecting an element that can be scrolled.

context: 'scroll-area',
name: 'scroll area'
switchOnSelectElement: el => el.classList.contains('hands-free-scrollable'),

Something to think about, what do you want to happen if there are no elements that can be scrolled? Maybe you can default to the entire body in that case?

I'll think about how you can ask the extension to render the allowed voice commands as a list at the bottom of the screen. I think the list would be [Up | Down | Exit]

freelyassociate

javi Thanks for the suggestions! I have added a name for the scroll-area context, as well as your fix for the scroll issue. The latest code is here: https://github.com/leekevinyg/handsfree-for-web-scroll-area-module/blob/master/src/index.js

As for the case where no elements can be scrolled, do you think it would make sense to just return the root context?

javi I'll think about how you can ask the extension to render the allowed voice commands as a list at the bottom of the screen. I think the list would be [Up | Down | Exit]

Ok sounds good. On a related note, would it be useful to show error messages in red on the bottom of the screen as well if possible? For example, another solution to the "no scrollable elements found" issue you mentioned above can be to show a red toast message on the bottom of the screen that says "no scrollable elements found", and then just return back to the root context. Just an idea! Let me know if you think it will be useful and I can help implement it with your guidance 🙂 Thanks again for you time.

javi

Glad to see the module is working 🙂

I think it'd be better to focus on the user's intention, for some reason he wants to scroll something, if we detect that there is no area to scroll, we can suppose that he wants to scroll the entire screen.

I just released a new version of the extension including the latest changes and the option to show the supported voice commands of the current context.
https://twitter.com/javipzv/status/1192092962362155009

You can tell the tool to show the supported commands returning the property showCommandList: true when executing an action (up and down), but if you do that you will see the command list only after executing some of these commands. What you really want is to see the list as soon as the context scroll-area is activated to do that you can use the setup hook.

i.e.

      context: 'scroll-area',
      switchOnSelectElement: el => el.classList.contains('hands-free-scrollable'),
      setup: () => ({ showCommandList: true })

This new feature will be supported by v3.4.0, hope Chrome Store will make it public in the following hours.

javi

Hi @freelyassociate, could you check the latest changes?

What do you think is the next step to implement? We can internationalize it and then include it within the supported modules so everybody can use the new commands.

Have a good day

freelyassociate

javi Hi Javi, sorry for the late reply! I have been busy with moving to a new place this last week. I will take your suggestion to scroll the main page when multiple elements are not present on the screen, as well as add in the showCommandList action. I will do this tonight and have it in by tomorrow morning 🙂

freelyassociate

Hi @javi, the latest changes are available at this link!

When no detectable scrollable elements are found we still go to the scroll-area context, and try to scroll the body. However in my testing this doesn't seem to be working on all pages, whereas the default "scroll down" command behaviour from the root context does work. Is it possible to trigger the default "scroll down" command behaviour from this line here?

Also, please advise on what I can do to help with internationalization to help get this module out to the public!

Additionally, I am not sure if this will be an issue when we release the module, but it seems that while the src/index.js file is able to be imported properly into the extension, trying to import the minified file under dist here results in Error when trying to import the module. Seems that module structure is invalid. Will this be an issue, and if so can you advise on how to fix it?

One last thing - I added an isScrollable helper to the dom-element-types tool and created a pull request here. Feel free to let me know if anything there needs to be fixed up!

Thank you 🙂

javi

I'm excited to see the progress here 🙂

I have posted a comment in your repository to fix the scroll issue of the body element and I reviewed the PR of isScrollable

I have published a repository including all the voice command modules that don't require access to the chrome extension permissions.
https://github.com/sljavi/handsfree-for-website-modules

At this moment I'm working on adding docs and tests to the repository, I hope the docs will be ready for Monday so you can read about the internationalization feature. In case you want to try to guess how does it work, as a short clue I can say that

module.name and module.description will be translated based on what module.i18n[selectedLang] defines.
module.context[index].actions[index2].name is translated based on what module.context[index].i18n[selectedLang] defines.
keep in mind that if a name must be translated it has to start with i18n-. i.e. 'i18n-scroll.command' otherwise, it isn't translated

My idea is to load the voice commands from two repositories "handsfree-for-website-modules" and "handsfree-for-web-modules". I'm also working on a new version of handsfree for web that can be loaded by any developer in a website, like jquery. As soon as it gets loaded allows the user to browse the page handsfree and also will allow the developer to add custom voice commands on the fly.

So in this new scenario,

Handsfree for web loads handsfree-for-website-modules and handsfree-for-web-modules
Handsfree for website loads handsfree-for-website-modules

I'll keep you posted,
Thanks!

freelyassociate

javi

javi I have posted a comment in your repository to fix the scroll issue of the body element and I reviewed the PR of isScrollable

Thank you, I updated the PR based on your comments! Just one more open question/issue here regarding the scroll issue with the body element.

I will start preparing a PR for the "scroll area" command in your new repository with the valid i18n strings. Is there anyway I will be able to test the translations before I make a PR? Thanks again.

javi

Hi freelyassociate

I just gave a try to the module and commented on Github my findings. I'm publishing a new version of HFW in this moment with the described issue.

At this moment I think the best way to test the module is by importing it within the extension. Hopefully this week I'll add a playground page to the new repository where everybody would be able to play with the voice command's modules.

I'll post here as soon as the docs are ready.

javi

freelyassociate I just have published the tests and docs for the new repository
https://github.com/sljavi/handsfree-for-website-modules

You can find the internationalization docs at the end of the readme, any feedback will be appreciated.

I'll work on the playground area an let you know

freelyassociate

javi

javi I just gave a try to the module and commented on Github my findings. I'm publishing a new version of HFW in this moment with the described issue.

Thank you! I incorporated your findings, as well as fixed a bug to not show the "element picker" if there is only one scrollable element. The changes are here. I will incorporate these changes and add internationalization tomorrow or the day after and make a PR on your new repository.

Btw, would it be possible to get this PR approved and merged and added into the tools parameter inside HFW, so it can be used in the new "scroll area" module? No worries if this is not possible at the moment.

javi

Morning @freelyassociate, I have approved and merge the PR. I publish it everywhere. Hopefully, in two days from now, the new version of it will be available in tools.

Let me know if you need some help with the i18n thing, it might be useful to check the scroll module mostly to see how the words are translated to Spanish and Portuguese. This is in case you don't speak some of those languages 😬

Thanks!

freelyassociate

javi Hi Javi, I have forked your repo and added the internationalizations here: https://github.com/leekevinyg/handsfree-for-website-modules/blob/master/src/scroll-area.js

I have created placeholders for the Spanish and Portuguese translations. I don't speak either of those languages and was going to try to translate them using the scroll module/google translate, but decided I better not in case I butchered them 😅. Would you be able to help fill them in? Thank you!