Spreadsheet cells in JavaScript for Google Docs - Sketch-n-Script

2020/03/13 — Mikaël Mayer - updated from my github blog post.

Have you ever wished you could synchronize content within one Google doc? Do you need temporary placeholders that you will replace later? Do you want to generate text? Do you miss spreadsheets in docs?

If you answered yes to one of these questions, this Google docs add-on this for you.
It enables you to define names (a.k.a. variables) anywhere in the document, and to insert computations that produce rich text anywhere in the document.

Define names

You can define a name anywhere in your document. It has to start with a word, the sign equal '=' and a non-empty space behind it. If there is an opening parenthesis, the content will be evaluated as JavaScript. If there is no opening parenthesis, the remaining until the end of the newline is treated as raw text. Definitions can depend on previous definitions.
The following are all valid definitions of names:

version = 1.0 beta
name = ("MySoft" + version)
citations = ({ mayer18: "https://dl.acm.org/citation.cfm?id=3276497", chugh16: "https://dl.acm.org/citation.cfm?id=2908103" })
cite = (function (name) { return typeof citations[name] !== "undefined" ? ["[" + name + "]", {link: citations[name]}] : "[" + name + "???]"; })

Write formulas in your document

Anywhere in the document, write the sign = followed by either

• a name, e.g.
  =version
• a name followed by a JavaScript expression in parentheses, e.g.
  =email("a@b.com", "Hello")
• a JavaScript expression inside parentheses or square brackets, e.g.
  =["p", {}, ["This is a paragraph"]]
=("Hello" + "world")
• a name followed by the sign @ and a javascript expression, to give a name to the expression, e.g.
  =name@("Hello")

For example, in a document, you would write raw

=name is out!
=[name, {bold: true}] is a n=(function() { var x = ""; var i = 10; while(i-- > 0) x = x + "i"; return x} ())ce =(/*s=*/"software") based on bidirectional evaluation =cite("mayer18") that goes beyond the original ideas of prodirect manipulation =s =cite("chugh16")

Render formulas

Open the menu "Formulas" and click on "Evaluate formulas". It replaces all formulas by their values. If you selected some text, only formulas in the selection would be evaluated. The document above would be rendered as:

MySoft 1.0 beta is out!
MySoft 1.0 beta is a niiiiiiiiiice software based on bidirectional evaluation [mayer18] that goes beyond the original ideas of prodirect manipulation software [chugh16]

API available

Inside paragraphs

Formulas used inside paragraphs in the document should either return raw text, or a 2-element array where the first element is the text, and the second is a Javascript object containing styling material. If the method setThing exists in this page, then thing : value is a valid style attribute. Here is a list of supported attributes and shortcuts

• backgroundColor (or background): a string representing a color like "#FF00FF".
• bold: boolean (true or false).
• fontFamily: string representing a valid font name.
• fontSize: integer representing a valid font size.
• foregroundColor (or color): a string representing a color like "#FF00FF".
• italic: boolean (true or false).
• linkUrl (or link): string representing an URL.
• strikeThrough (or strike): boolean (true or false).
• textAlignment (or align): either "normal", "superscript" or "subscript"
• underline: boolean (true or false).

Stand-alone formulas

It is also possible to have formulas that generate paragraphs, list items, headings and tables. In this case, the formula should span an entire paragraph. Here are the available functions:

• Paragraphs, e.g.
  =["p", {}, ["This is a paragraph"]]
• Titles, e.g.
  =["p", {heading: "heading1"}, ["Hello ", ["world", {italic:true}]]]
=h2("Hello world")
• Tables, e.g.
  =["table", {}, [["x", "1", "2"], ["1", "1", "2"], ["2", "2", "4"]]]
• List items, e.g.
  =["li", {id: 1, nesting: 0, glyph: "number"}, "Todo item"]
=li("Todo item")

You can always consult the cheat sheet which is at the bottom of the extension.

Visualize where formulas are being used

To make sure a field is computed and not just text, click on "highlight values" and then click on "Display values". It highlights all places where the text is computed. If you selected some text, only computed text in the selection would be highlighted. The document above would be transformed to (colors should be background colors, but I cannot do this in WordPress):

MySoft 1.0 beta is out!
MySoft 1.0 beta is a niiiiiiiiiice software based on bidirectional evaluation [mayer18] that goes beyond the original ideas of prodirect manipulation software [chugh16]

You can hide these highlights by unchecking the box "Hightlight values" and click on "Display values". If you selected some text, only highlights in the selection would be removed.

Reveal formulas

It can be useful to change formulas. To do so, check "highlight formulas" and click on "Display formulas" . It will replace all computed fields by their respective formulas and highlight them in orange. If you selected a portion of the text, only the formulas in the selection would be revealed. You can then edit any formula:

=name is out!
=name is a n=(function() { var x = ""; var i = 10; while(i-- > 0) x = x + "i"; return x} ())ce =(/*s=*/"software") based on bidirectional evaluation =cite("mayer18") that goes beyond the original ideas of prodirect manipulation =s =cite("chugh16")

Once you're done, click on "Formulas" and then "Display values" to replace the formulas by their values.

=name is out!
MySoft 1.0 is a niiiiiiiiiice software based on bidirectional evaluation [mayer18] that goes beyond the original ideas of prodirect manipulation software [chugh16]

Availability

This extension is available as a Google docs add-on that can be added to any document by opening it, open "Add-ons" and click "Add add-on", and look for "Formulas for Google Docs".

Wish list

There are many features one would like from this extension:

• Modify a formula or a name's value by modifying its output anywhere. It works for variables, paragraphs, tables, list items, and text constant only for now. One day, it will work even for functions.
• Reading data from spreadsheet or other APIs
• Keyboard shortcuts

They might be coming soon. If you want them sooner,

Copy-and paste unfortunately cannot be supported due to how formulas are stored. However, you can reveal a particular formula to duplicate it.

This work was supported by Swiss National Science Foundation Early Postdoc.Mobility Fellowship No. 175041

Privacy policy is just below

---------------------------

Privacy Policy

Mikaël Mayer built the Spreadsheet cells in JavaScript for Google Docs - Sketch-n-Script app as an Open Source app. This SERVICE is provided by Mikaël Mayer at no cost and is intended for use as is.

This page is used to inform visitors regarding my policies with the collection, use, and disclosure of Personal Information if anyone decided to use my Service.

If you choose to use my Service, then you agree to the collection and use of information in relation to this policy. I am not collecting personal information in any way, except if you click on the survey and fill out information to contact me.

The terms used in this Privacy Policy have the same meanings as in our Terms and Conditions, which is accessible at Spreadsheet cells in JavaScript for Google Docs - Sketch-n-Script unless otherwise defined in this Privacy Policy.

Information Collection and Use

While using our Service, I will not require you to provide us with certain personally identifiable information. The information that is collected from the document will be retained on your document and is not collected by me in any way.

The app does not use third party services that may collect information used to identify you.

Log Data

I want to inform you that whenever you use my Service, in a case of an error in the app, I do not collect data and information. Submitting error is a voluntary step you can do yourself at this link.

Cookies

Cookies are files with a small amount of data that are commonly used as anonymous unique identifiers. These are sent to your browser from the websites that you visit and are stored on your device's internal memory.

This Service does not use these “cookies”.

Service Providers

I may employ third-party companies and individuals due to the following reasons:

• To facilitate our Service;
• To provide the Service on our behalf;
• To perform Service-related services; or
• To assist us in analyzing how our Service is used.

I want to inform users of this Service that these third parties do not have access to your Personal Information.

Security

I value your trust in installing the App. But remember that no software is 100% safe, use it at your own risks and make back-ups regularly to avoid any error.

Links to Other Sites

This Service may contain links to other sites. If you click on a third-party link, you will be directed to that site. Note that these external sites are not operated by me. Therefore, I strongly advise you to review the Privacy Policy of these websites. I have no control over and assume no responsibility for the content, privacy policies, or practices of any third-party sites or services.

Children’s Privacy

These Services do not address anyone under the age of 13. I do not knowingly collect personally identifiable information from children under 13. In the case I discover that a child under 13 has provided me with personal information, I immediately delete this from our servers. If you are a parent or guardian and you are aware that your child has provided us with personal information, please contact me so that I will be able to do necessary actions.

Changes to This Privacy Policy

I may update our Privacy Policy from time to time. Thus, you are advised to review this page periodically for any changes. I will notify you of any changes by posting the new Privacy Policy on this page. These changes are effective immediately after they are posted on this page.

Authorizations required

Just a comment on the authorizations this extension requires, to ensure it conforms to Google docs Policy. This extension requires two sensitive authorizations.

• Permission to access the UI of Google docs
• Permission to communicate with an external server.

The first permission is straightforward, since the extension's purpose is to display a sidebar to trigger the execution of JavaScript formulas, and switch from/to formula to/from value display.

The second permission is more subtle. Fortunately, the project is open-source , so you can inspect that the only place when invoke UrlFetchApp, it's on Code.js line 1287. The variable url there is actually provided by users of the extension themselves, and is in no way invoked by the extension in a way to monitor your usage.

Being able to fetch URL means that it makes it possible to
1) Generate images from URLs, e.g. LaTeX formulas as highlighted in the example.
2) Fetch custom content within your Google doc. For example, if you write the following in your Google doc

test = (UrlFetchApp.fetch(“http://www.mocky.io/v2/5ca3eb514b00005600209882”).getContentText())

=test

After execution, you'll see "Hello world", which is the text that I saved in mocky.io at this particular URL. You can use your own.

How does this application accesses, uses, stores, or shares Google user data?

This application does not access your Google user data. Authentication is only required to access the app, but the app is open-source, you can check it here and by this post I claim that this is the exact source code.

The only data this application stores is by using the APIs DocumentProperties, UserProperties and ScriptProperties to store formulas and reminders to fill the evaluation on Google form if you did not discard them.

Contact Us

If you have any questions or suggestions about my Privacy Policy, do not hesitate to contact me at sketch-n-sketch@googlegroups.com.

My volunteering organization's manager asked me if I could print organizational charts to display who is in which position in a hierarchical way (150+ positions). That way, it would help him to see which positions are vacants, to understand who has been there for a long time, and so on.

I accepted the challenge.

It took me 2h30 to find how to scrap the data, and 3h to write a converter using  Editor.  a new tool I developed at the University of Chicago with an awesome team to dynamically create data-based documents in a very flexible way. Editor enables  both the edition of the code as well as the edition of the final document using advanced bidirectional evaluation techniques, and runs as an http server.

In this article, I will report on my experience using Editor (as of today, January 16th, 2019), to solve the problem above.
The example is available for download there and the final solution is in a separate branch. To run and tweak the example, after installing Editor, run editor --openbrowser inside the folder.

Challenge accepted

Organizational charts were a perfect fit for Editor. I was first able to scrap the organization's data and pictures associated to the positions in our local organization. Basically, the input data was

type Input = List (Posision, (Name, StartDate))

where the Position, Name and StartDate are all aliases of strings.
As an example, such data could have been:

input = [
  ("Project manager", ("Ravi Chugh", "2013")),
  ("Research manager", ("Mikael Mayer", "2017")),
  ("Research Apprentice", ("Mikael's son", "2018")),
  ("Research manager2", ("Brian Hempel", "2015"))]

My manager already had made organizational charts last year using PowerPoint, but they were quickly outdated since many people moved in and out. He was tired of updating them by hand. He handed me those charts. I looked at them. They also contained Position, Name, the picture, and the starting date, but in a hierarchical way, some positions appearing multiple times.
These organizational charts consisted of a leader that supervises many leadees, which might be in turn leaders for other sub-organization charts. I decided to encode a chart using the following type:

type OrgChart = Leader Position (List OrgChart)

An example of chart could be the following:

chart =
  Leader "Project manager" [
    Leader "Research manager" [
      Leader "Apprentice" []
    ],
    Leader "Research manager" []
  ]

Then I created a recursive function that would transform an OrgChartto an HTML element <div> , including the sub-charts, looking up the position name into the input data to fill the name, picture and start date, and voilà, after some css, I got printable organization charts:

Well, something is wrong, obviously. The usual way to fix this problem is to fix the tree in the code so that the positions match the inputs (Research Apprentice instead of Apprentice, or Research Manager2 instead of Research Manager for the second time). But at the same time, we still want to keep these shorter names in the organization chart.

Editor's unique features

Editor made the tweaking process much easier.

• First, Editor enables me to rename the positions directly on the view depicted by the screenshot above. For example, after renaming  right next to my second picture 'Research Manager' to 'Research Manager2', the picture and the name update automatically and we get the following chart:
  
• However, it's not useful to display the "2" of "Research Manager2". We want to display "Research Manager" anyway. The way to do this is to normally create a list of renamings:

  renamings = [("Research Manager2", "Research Manager")]

  and use this list to compute a display name using the code below:

  listDict.get position renamings
  |> Maybe.withDefaultReplace (freeze position)
  (\posDisplayName -> -- Do whatever with the display name)

  This solves the problem above.
  Even better, if I now visually rename my position to "Researcher", it automatically adds the line ("Research Manager", "Researcher") to the list renamings!
  Let me explain this step by step. When displaying my position, position equals "Research Manager". the first line thus returns Nothing since there is initially no renaming, and the second line returns "Research Manager" which is assigned to posDisplayName that is used to display the name in the viewport.
  If I now visually rename my position, let's say to "Researcher", the second line back-propagates Just "Researcher" back to the first line, which in turn -- since it returned Nothing previously -- adds the line ("Research Manager", "Researcher") to the renamings. The power of lenses! It was very useful to rename 150+ positions in context.

• Now, if I corrected the position "Apprentice" by renaming it to "Research Apprentice", it might create such a renaming. Whatever, if the position is not found, I do not use the variable posDisplayName but the variable position so that modifications are not renamings for display but really modify the organizational chart itself, until it finds a position match.
• When a position in the organization chart was not found in the source data, I looked for positions in the source that could match and first displayed them when I hovered the unknown positions as suggestions on how to rename them.
  
  But Editor also provides an API of buttons that, when triggered, can "modify variables" on back-propagation. So I did. Given a suggestion that I computed, I created the button:

  Html.button suggestion "Is this the correct position name?"
          position (\oldPosition -> suggestion)
  

  With 1 click I can now accept a suggestion that would replace the previous value of position in the organigram tree itself.
  
  After clicking, we obtain the following:

  

• Even though this is cool, I started to get tired of first clicking on a button to accept a suggestion, and then renaming  the new position to match the position nickname what was in the original paper organization chart.
  Oh, but this is easy, I just needed to not only modify the position name in the organization tree, but also directly add a renaming!

  Html.button suggestion "Is this the correct position name?"
   (position,       renamings) (\_ ->
   (suggestion, renamings ++ [(suggestion, position)]))

  Therefore, the previous step would have kept the display name "Apprentice".
  This saved me tons of time and ensured that the organization tree was consistent with the renamings.

• Finally, I needed to add and remove some people around. Instead of going back to where I defined the chart, I found out that I could also integrate in the display two buttons, one to add a new child to the tree, and another one to remove all children:

  Html.button "+" "Add a child" children
      ((++) <| [Leader "TODO" []])
  Html.button "-" "Remove children" children (always [])

  One of the interesting points found nowhere in other programming models is that I don't need to define how to walk the tree to insert the new Leader: Because the updates evaluate the program in reverse, it will insert it at the correct location in the tree.
  The final interface looked like:
  

  And of course, I added a CSS media query for printing that removed all the buttons when I print the page

  <style type="text/css" media="print">
  button {
   display: none !important;
  }

So far so good. But not everything was beautiful.

What Editor caused me pain for

Editor is still experimental and academic. I found many bugs and performance bugs while using it, that provide the hint that much more work is needed before it can get widespread adoption. Here is a small list:

• Editor does not work well with huge amounts of data. If I display 20 organization charts at the same time (more than 150 people) and try to update, after 2-3 times I encounter memory exceptions. I had to develop organization charts by commenting the ones that I created before.
• I discovered bugs while performing multiple updates at the same time that crashed the system. Fortunately, all the files were always written on the file system so I did not experience any loss..
• Editor is slow to update when the file gets larger and larger. It was not unusual for me to wait 20 seconds to delete a simple character.
• Furthermore, I had to wait that the previous modifications were taken into account before making new ones, which really slowed me down sometimes. I went back to code editing multiple times instead of sticking to visual edition.
• Many times, I want to modify attributes or styling, but I had no other way than going to devtools or edit the code itself. Having a contextual style and attribute editor would have make my life much easier.

What Editor is good at

The reason why I chose Editor instead of a WYSIWYG editor like Smartdraw is that I feel more comfortable in programming things I already know (HTML, Javascript, CSS) rather than a new file format with tons of APIs. To sum up, for this use case, Editor was good at:

• Writing any kind of dynamically generated HTML page and back-propagate any modifications back to the source code
• Letting you edit the page using the DOM inspector or elements using the contenteditable attribute
• Resolving ambiguities by asking questions (are you trying to modify the template, the plug-in or the underlying data?)
• Letting you add custom editing buttons
• Keyboard shortcuts to save the page

Editor has many other features that are promising for other kinds of applications

• It lets you edit links, drag pictures where you need them (like google docs, but on any page)
• It starts a server in which you can not only browse directory, but also rename and delete files
• It renders Markdown files but lets you edit them visually
• It supports HTTPS to create secure servers, and experimental authentication
• It is compatible with Grammarly, Ace editor, Google Analytics, Google Sign-in, and provides an API to make pages compatible with other plug-ins that otherwise blindly modify the page
• It can open desktop files like a real application in Windows, it's just that it launches a temporary server and opens a tab in your browser

Conclusion

I still think Editor was the most adapted tool to develop these organization charts, and I am looking forward using and developing the new version of Editor. Oh, and by the way, Editor is open-source and welcomes contributions. Also, follow me on Twitter to know about the next releases.

There is a Pull Request that highlights the interface elements that Editor made possible and that I described in this post.

--
This work was supported by Swiss National Science Foundation Early Postdoc.Mobility Fellowship No. 175041

Today, I gave the Mill build tool a try and I am very enthusiastic to see it work very quickly for our Elm project Sketch-n-Sketch. Below, I describe the old Makefile and then the Mill equivalent.

As for many other build tools, Mill keeps tracks of dependencies between tasks. Except that tasks can be simply defined as pure functions, and that's the power of Mill.
Furthermore, Mill can watch sources and provides an easy access to the type-proof Ammonite shell commands, make it suitable for a general-purpose build tool.

The original Makefile whose absolute path was ./src/Makefile looked like this:

ELMMAKE=elm-make

all:
 $(ELMMAKE) Main.elm --output ../build/out/sns.js

html: all
 $(ELMMAKE) Main.elm --output ../build/out/sns.js
 cp Native/aceCodeBox.js ../build/out/
 cp Native/aceTooltips.js ../build/out/
 cp Native/animationLoop.js ../build/out/
 cp Native/fileHandler.js ../build/out/
 cp Native/deucePopupPanelInfo.js ../build/out/
 cp Native/proseScroller.js ../build/out/
 cp Native/dotGraph.js ../build/out/
 cp Native/colorScheme.js ../build/out/
 cp Native/keyBlocker.js ../build/out/
 cp ../ace-builds/src/ace.js ../build/out/
 cp ../ace-builds/src/mode-little.js ../build/out/
 cp ../ace-builds/src/theme-chrome.js ../build/out/
 cp ../viz.js/viz.js ../build/out/
 mkdir -p ../build/out/img
 cp ../img/sketch-n-sketch-logo.png ../build/out/img/
 cp ../img/light_logo.svg ../build/out/img/
 cp ../img/*.png ../build/out/img/

A few notes about this makefile. There are two targets, all and html. html is complete in the sense that it not only compiles the Elm files, but it also copies the javascript files necessary to the final application, as well as other files. Also, we wanted to copy these files only when needed, mostly after a successful compilation.

Following the excellent Mill's documentation, I converted the above Makefile to a top-level ./build.sc containing :

import mill._, ammonite.ops._

val ELM_MAKE = "elm-make"

object SNS extends Module {
 def millSourcePath = pwd
 implicit def src: Path = pwd / 'src

 def sourceRoot   = T.sources { src }
 def nativeRoot   = T.sources { src / "Native" }
 def allSources   = T { sourceRoot() ++ nativeRoot() }

 val outDir = pwd/'build/'out

 def all = T{
   allSources()
   stderr( %%(ELM_MAKE,"Main.elm", "--output", outDir/"sns.js"))
 }

 def copyNative = T{
   nativeRoot()
   all() match {
     List("aceCodeBox.js",
          "aceTooltips.js",
          "animationLoop.js",
          "fileHandler.js",
          "deucePopupPanelInfo.js",
          "proseScroller.js",
          "dotGraph.js",
          "colorScheme.js",
          "keyBlocker.js"
     ).map(src/'Native/_).foreach(copy(_, outDir))
     List("ace.js",
          "mode-little.js",
          "theme-chrome.js"
     ).map(pwd/"ace-builds"/'src/_).foreach(copy(_, outDir))
     copy(pwd/"viz.js"/"viz.js", outDir)
     mkdir ! pwd/'build/'out/'img
     copy(pwd/'img/"light_logo.svg", outDir/'img)
     ls ! pwd/'img |? (_.ext == "png") |! (copy(_, outDir / 'img))
 }

 def html = T{
   copyNative()
   all() match {
   case Left(msg) =>
     System.out.print("\033[H\033[2J"+msg)
     false
   case Right(ok) => true
   }
 }


 def copy(file: Path, outDir: Path) = {
   val out = outDir/file.last
   if (exists! out) rm(out)
   mkdir! outDir
   cp(file, out)
 }

 def stderr(commandResult: =>CommandResult): Either[String, String] = {
   try {
     Right(commandResult.err.string)
   } catch {
     case ammonite.ops.ShelloutException(commandResult) =>
     Left(commandResult.err.string)
   }
 }
}

def html = T{ SNS.html() }

It's not completely straightforward to do this transformation, but I was able to do it in less than 2 hours. Not bad for a first-time usage of Mill I hope 🙂
I changed the order of the build file, so that html now depends on the files being copied.
Here were some necessary tweaks to make this magic happen:

• %% requires an implicit path in scope, which I define at the beginning of the object.
• %% launches a process but throws an exception if the exit code is not zero. I prefer to catch this exception using an Either type and return the standard error instead (after of course cleaning the screen using the System.out.print("\033[H\033[2J") command)
• I needed to specify a return value for the task html. If I omitted the true/false, the type checker might infer the return type "any" for which there is no pickler, i.e. a way to nicely format the data. It is likely to be useful to return a boolean if other future tasks depend on html.
• Ammonite's default methods to copy files are not sufficient for what we need here, which is to overwrite a file into a folder. I thus created a wrapper for that, to make the syntax nice.
• I wrapped all the tasks into a module, but this was completely optional, I could have had a flat file instead. To make the task html appear top-level, I just creates a reference to it instead.

The magic happens:

• Thanks to Mill's task dependency feature, if we don't change the native files, they are not copied anyway, which saves ~1s of build compared to the original Makefile. Excellent!
• It's very easy to refactor common variables (e.g. outDir)
• Tasks can be much more easily composed than with other build tools. For example, the allSources is a task that is executed only when one of the two source directories change.
• By executing ./mill -watch html Mill can recompile the sources as soon as they are modified. That's great!

I conclude that Mill 0.2.2 passed the test of replacing Makefile and is not only useful for Scala projects, but for general-purpose projects. Mill seems to be the right way of what a build file should be.

It's very uncommon to migrate from Scala to another language. Yet this happens. It happened to me.

During a second job interview, I was asked in advance not to code in Scala but only Java or Python. Certainly because this is what the interviewers understood. Even if I would push a team to start using Scala if I were hired, I cannot push my interviewers to use Scala ...

"Hey please make my interview easier, learn scala, and come back to me".

Instead I had to prove them that I was able to code in Java. That way, It would be much easier to defend switching to Scala later.

Below is the cheat sheet that  I developed in order to be more productive during my interview. It really helped me a lot. Hoping it saves you time too !

Happy coding !

Int, Long, Short, Char,
Byte, Boolean, Double

int, long, short, char,
byte, boolean, double

(a: A, b: B) => f(a)

(A a, B b) -> f(a)

(a: A, b: B) => {
   F
   f(a)
}

(A a, B b) -> {
  F;
  return f(a);
}

var x: A = ...

A x = ...;

val x: A = ...

final A x = ...;

var x = new Array[Int](15)
x(0) = 2
val a = x[2]

int[] x = new int[15]
x[0] = 2;
int a = x[2];

var x = Array(1, 2, 3)

int[] x = { 1, 2, 3 };

var m = List(4, 2, 3)

m =  m.sortBy(i => i*i-4*i)

List<Int> m = Arrays.asList(4, 2, 3;
Collections.sort(m, (Int i) -> i*i-4*i);

def theMethod(a: String, b: A => Unit) {
  val theA: A = convert(a)
  b(theA)
}

void theMethod(String a, Block

theMethod(“test”,
  (a: A) => println(“true”))

theMethod(“test”,
  (A a) -> println(“true”))

for(i &lt collection) {

}

for(Elem i: collection) {

}

val a: Photo => Unit
a(photo)

Consumer<Photo> a
a.accept(photo)

val  a: Photo => Boolean
a(photo)

Predicate<Photo> a = … ;
a.test(photo)

val a: X => Y
a(x)

Function<X, Y> a = …;
a.apply(x)

val photos = List(...)
val output = photos.filter(p => p.sizeKb < 10)

List<Photo> photos = 
  Arrays.asList(...)
List<Photo> output = photos.filter(
  (Photo p)-> p.getSizeInKb()<10);

val photos = photos.filter(p => p.sizeKb > 10000)
            .map(p => p.name)

List<Photo> photos = Arrays.asList(...)
List<Photo> output = photos.filter(
  (Photo p) -> p.getSizeInKb() > 10000)
  .map(p -> p.name);

def heavyComputation = (0 to 10).toList.par.foreach(
  i => heavyComputation(i))

IntStream.range(0,500).boxed()
  .collect(Collectors.toList())
  .parallel().forEach(
  (int i) -> heavyComputation(i))

object Sex extends Enumeration {
  val MALE, FEMALE = Value
}

public enum Sex {
    MALE, FEMALE
}

case class Person(age: Int, name: String)

class Person {
  int age;
  String name;

  Person(int _age, String _name) {
    age = _age
    name = _name
  }
}

trait A {
  def methodToOverride(b: B): C
}

public interface A {
  C methodToOverride(B b);
}

def methodTest(a: A) = ???
methodTest(new A {
  def methodToOverride(b: B) = convertToC(b)
})

methodTest((B b) -> convertToC(b));

.toStream

.stream()

.foreach(lambda)

.forEach(lambda)

def m() = {
  var x = 2
  x = 3
  val consume = (i: Int) => println(i + x)
  consume(5)
}

void m() {
  int x = 2
  int y = 3 // Different variable !
  val consume = (i: Int) => println(i + y)
  consume(5)
}

def m(a: A*) { }

def m(A... a) {}

trait A {
  val log: String => Unit
  def debug(msg: String) = { 
    log(msg)
  }
}

public interface A {
  Consumer<String> log
  void debug(String msg) default { 
    log.accept(msg);
  }
}

month match {
  case 1 => “January”
  case _ => “Invalid month”
}

switch (month) {
    case 1:  monthString = "January";
         break;
    default: monthString = "Invalid month";
                     break;
}

object A {

}

class A {
  static A instance = new A
  private A() {
  }
}

val P= “a(b*)”.r
P.findFirstIn(“abbacabbbbd”).map{
  m =>
   println(s”I found the text${m.group} starting at ${m.start} and ending at index ${m.end} \n”)
}.getOrElse(println(“No match found”))

Pattern P = Pattern.compile(“a(b*)”)
Matcher matcher = P.matcher(“abbacabbbbd”);
boolean found = false;
while (matcher.find()) {
    console.format("I found the text" +
        " \"%s\" starting at " +
        "index %d and ending at index %d.%n",
        matcher.group(),
        matcher.start(),
        matcher.end());
    found = true;
}

val res = for(i 

List<CT> res = A.filter(pred)
  .flatMap((int j) -> B(j))
  .map((K j) -> C(j));

val myNums: ArrayBuffer[T forSome {T 

List<? extends Number> myNums = new ArrayList<Integer>();

val myNums: ArrayBuffer[T forSome {T >: Number}] = ArrayBuffer()

List<? super Number> myNums = new ArrayList()

def copy[U : Number](source: ArrayBuffer[U],
               destiny ArrayBuffer[V]) {
   for(number 

public void copy(List<? extends Number> source,
                        List<? super Number> destiny) {
   for(Number number : source) {
      destiny.add(number);
   }
}

var s = Set[A]()
s += new A
s(a)
s.isEmpty
for(i 

HashSet<A>

var m = Map[A, B]()
m += a -> b

HashMap<A, B> m = new HashMap<A, B>()
m[a] = b

val m = new concurrent.collection.HashMap()

Map m = Collections.synchronizedMap(new HashMap(...));

val s = m.toSet

Set<Map.Entry<A, B>> s = m.entrySet

val t = (“test”, 2)
t._1
t._2

java.util.Map.Entry t = new java.util.AbstractMap.SimpleEntry<>(“test”, 2)
t.getKey()
t.getValue()

Ever wanted to iterate over Android cursor in the scala way ? This article will explain how with scala 2.11 we can write a library to do this awesome calls.

import AugmentedCursor._

val uri =
  Uri.parse("content://com.android.calendar/events")
val cursor = getContentResolver().query(uri,
  Array("calendar_id", "title", "description"),
  null, null, null)
for((calendar_id, title, description) <-
     cursor.toIterator[Int, String, String]) {
   /// ... Do something with calendar_id, title
   ///     and description for each entry
}

This is so much better than writing complicated code, and besides, it provide types  for the iteration variables !

Here is how we can make this magic happen.

First, we need to create an implicit class so that the method .toIterator[Int, String, String] can be defined.

object AugmentedCursor {
  implicit class RichCursor(c: Cursor) {
    def toIterator[T, U, V] = ???
  }
}

Now, the method toIterator should return an Iterable[(T, U, V)] to be able to write the initial code. Thus:

implicit class RichCursor(c: Cursor) {
  def toIterator[T, U, V] = new Iterable[(T, U, V)] {
    def iterator = ???
  }
}

The iterator itself must be an iterator over the triplet. Since we will only call it once and then close it, we can inline its construction.

implicit class RichCursor(c: Cursor) {
  def toIterator[T, U, V] = new Iterable[(T, U, V)] {
    def iterator = new Iterator[(T, U, V)] {
      def hasNext: Boolean = ???
      def next: (T, U, V) = ???
    }
  }
}

The hasNext method can compare the position of the cursor against the length of the result. Since during iterations, hasNext is called only once per iteration, we can close the cursor when there is nothing more to fetch.

def hasNext: Boolean = {
  val res = c.getPosition < c.getCount - 1
  if(!res) c.close()
  res
}

The method next is as simple, it will move the cursor to one place and then extract the right positions.

def next: (T, U, V) = {
  c.moveToNext()
  ???
}

Here we have a new problem. There is no such method as c.get[T](0) (take the first element of the cursor, and cast it to T). To the contrary, there are some methods such as c.getString, c.getLong, c.getInt, etc.

This is the time to set up a constraint: T, U and V can only be of type that the cursor can extract. We can enfore this by requiring that there exist implicit objects which can extract the type T out the cursor. For that, we define the trait Iter and two implicit objects implementing it, in the top-level object:

trait Iter[T] { def apply(c: Cursor, i: Int): T
implicit object IterS extends Iter[String] {
  def apply(c: Cursor: i: Int):String = c.getString(i)
}
implicit object IterI extends Iter[Int] {
  def apply(c: Cursor: i: Int):Int = c.getInt(i)
}

Nice, but how to be able to include them now in the next function? Simply, we can use implicitly[Iter[T]] to retrieve the implicit object and call its apply method. Sorry, we cannot use the syntactic sugar of removing the word "apply" since the method implicitly does not take parameters and this would be ambiguous. Thus:

def next: (T, U, V) = {
  c.moveToNext()
  (implicitly[Iter[T]].apply(c,0),
   implicitly[Iter[U]].apply(c,1),
   implicitly[Iter[V]].apply(c,2))
}

Now the compiler complains that there is no guarantee that there is an implicitly[Iter[T]].  How can we ensure that so that T is restrained? There is a funcitonality in scala: implicit parameters.  If the compiler can resolve them, no need to specify them ! This means that the method toIterator[T, U, V] introducing the three  type parameters requires the implicit objects to work, that is:

def toIterator[T, U, V](implicit objT: Iter[T], objU: Iter[U], objV: Iter[V])

There is also an even better syntax for that in scala, syntactic sugar:

def toIterator[T: Iter, U: Iter, V: Iter]

What's next? Abstraction ! We did it for three parameters, but perhaps we want to have one, two or more ! The refactoring is left as an exercise for the reader. For the impatient reader, the solution is below:

You need to  to get the syntactic sugar above. Voilà !

// AugmentCursor.scala
import android.database.Cursor
object AugmentedCursor {
  trait Iter[T] {
    def apply(cursor: Cursor, index: Int): T
  }
  implicit object IterInt extends Iter[Int] {
    def apply(c: Cursor, i: Int) = c.getInt(i)
  }
  implicit object IterString extends Iter[String] {
    def apply(c: Cursor, i: Int) = c.getString(i)
  }
  implicit class RichCursor(c: Cursor) {
    private def implicitIterator[T](f: => T) = new Iterator[T] {
      def hasNext = {
        val res = c.getPosition < c.getCount - 1
        if(!res) c.close()
        res
      }
      def next() = {
        c.moveToNext()
        f
      }
    }
    private def implicitIter[T](f: => T) = new Iterable[T] {
      def iterator = implicitIterator[T] {
        f
      }
    }
    def toIterator[T : Iter] = implicitIter {
      implicitly[Iter[T]].apply(c, 0)
    }
    def toIterator[T : Iter, U: Iter] = implicitIter {
      (implicitly[Iter[T]].apply(c, 0),
       implicitly[Iter[U]].apply(c, 1))
    }
    def toIterator[T : Iter, U: Iter, V: Iter] = implicitIter {
      (implicitly[Iter[T]].apply(c, 0),
       implicitly[Iter[U]].apply(c, 1),
       implicitly[Iter[V]].apply(c, 2))
    }
 }

Besides the lessons (for functions) given by Sebastian, the author of Scala-js, here are some more lessons I learned while porting javascript code to Scala to use Scala-js.

• Wrap your code. First, wrap your code in an object's body. You need this since scala does not authorize code outside of objects or classes. Extend JSApp to benefit from variables such as document, window, etc.
  object YourApp extends JSApp { ... }
  Put your code inside the brackets.
• Object literals: To convert the following javascript snippet:
  { action: "bind", data=42, check=true }
  you can use the literal import and use one of the following two constructions:

  import js.Dynamic.{literal => l}
  l(action="bind", data=42, check=true)
  l("action"->"bind", "data"->42, "check"->true)

• Order of val/var/def: The order of var/val and defs inside an anonymous function matter. You will have to correct all back-references. Hence, the following:

  function changeValue() { value = 5; }
  var value = 0

  which is valid in javascript, will has to be rewritten as

  var value = 0
  function changeValue() { value = 5; }

• Implicit converters from Dynamic to Boolean: Because js.Dynamics are sometimes used in boolean conditions or for comparison, I added implicit converters so that it does not throw errors.

  object Implicits {
    implicit def dynamicToBoolean(d: js.Dynamic): Boolean = d.asInstanceOf[Boolean]
    implicit def dynamicToString(d: js.Dynamic): String = d.asInstanceOf[String]
  }

• Warning comparing dynamic with strings: To remove the annoying warnings which arrive when you compare dynamic with strings, you can add the following implicit:

• object Implicits {
    implicit class ComparisonOp(d: js.Dynamic) {
      def ==(other: String) = d.asInstanceOf[String] == other
    }
  }

• Use of the javascript "in" keyword
  if(key in obj) has to be translated to if(!js.isUndefined(obj(key)))
• for...in
  for(a in obj) has to be translated to:
  for(a <- js.Object.keys(obj))
• String extraction: Instead of extracting chars with
  string(index)
  prefer the following
  string.substring(index, 1)
  which returns a String instead of a Char. It is hard to convert from Char to String.
• Zero-argument functions: Careful when you translate zero-arg functions to def. If you pass them as parameter, you need to add the modifier _ so that they are not evaluated. For example:

  function loadSomething() { ... }
  $("#button").change(loadSomething)

  has to be translated to:

  def loadSomething() { ... }
  $("#button").change(loadSomething _)

• setTimeout:  You will have to translate setTimeout using another construct, with a lazy evaluated block. Thus, this:
  setTimeout(function() { ... }, 999)
  becomes:
  js.timers.setTimeout(999){ ... }Do not put anonymous function inside the brackets like { () => ...} else they the function body will never be evaluated !
• console.log: In order to use console.log directly in your code, import the following which provides a statically typed object

  import org.scalajs.dom._
  console.log("Whatever")

  or alternatively, you can use the dynamic global variable:

  import js.Dynamic.{global => g}
  g.console.log("Whatever")

• Call object's main method. Last but not least. If you define your methods in an object, especially some jQuery onLoad event, you need to call at least one method from the object for it to be initialized. Indeed, objects are lazily created.
  For example, if your initializer object is:

  package your.app
  import scala.scalajs.js.JSApp
  import org.scalajs.jquery.{jQuery => $}
  object YourApp extends JsApp {
    $(document).ready(() => { ...
     })
   def main() = {}
  }

  you need to include a script in your html like this:
  <body onload="your.app.YourApp().main()">

• Bracket access on custom objects. For the call point["name"] with objects like the following, add a method for custom bracket access. You may factorize this method in a custom trait.

  val point = new Object {
    val name = "Mikael"
    val age = 18
    @JSBracketAccess
    def apply(index: String): js.Dynamic = js.native
  }
  //point("name") will retrun "Mikael" as a js.Dynamic

• String-to-int conversion:
  • If you use JQuery and  observe $("something").value()*1, since  value() returns a js.Dynamic, simply converting this to $("something").value().toInt will fail at run-time (the toInt method is not defined on object of type string). However, you can do the following:
    $("something").value().asInstanceOf[String].toInt
  • If you find the pattern 1*s , where s is a string which might be null, don't use s.toInt directly in scala. It fails if s is null, whereas 1*null  == 0 in javascript. Instead, do the following:

    implicit class OrIfNull[T](s: T) {
      def orIfNull(e: T): T = if(s == null) e else s
    }
    s.orIfNull("0").toInt

Remember, next time, to directly start with scala-js 🙂

After integrating basic scalajs support into the project (part 1), we now want to include the following libraries (described here):

• dom for manipulating the DOM structure of the HTML page
• jquery to have an easier access to the DOM. And of course, we want to use the shortcut "$" instead of the verbose jQuery

To do so, add the following settings to your mainprojectfolder/build.sbt file describing the js project defined in the last article.

lazy val js = (project in file("js")).enablePlugins(ScalaJSPlugin).settings(
  libraryDependencies += "org.scala-js" %%% "scalajs-dom" % "0.8.0",
  libraryDependencies += "be.doeraene" %%% "scalajs-jquery" % "0.8.0"
)

This is the safest way to ensure that the libraries are imported.
Now in your scala file compiling to javascript (myscript.scala), you can include these import statements:

package your.package
import org.scalajs.dom
import dom.document
import org.scalajs.jquery.{ jQuery => $ }
import scala.scalajs.js.annotation.JSExport

object YourApp extends JSApp {
  @JSExport def addClickedMessage(): Unit = {
    val message = "You clicked the button!"
    //appendPar(document.body, message) (DOM version)
    $("body").append(s"<p>$message</p>")
  }
 
  def main(): Unit = {
    println("Hello world!")
  }
}

Now in your html you can include something like this, and this will work:

<button id="click-me-button" type="button" onclick="your.package.YourApp().addClickedMessage()">Click me!</button>

I hope that you will enjoy Scala-js as much as I do !

It can be useful to get rid of Javascript and switch to a better typed language such as Typescript or Scala. Both compile to Javascript.

In this blog post, I will describe how work with scala-js to transform the existing javascript files of a project to scala and have the compilation pipeline work as usual.

Assuming you are working with sbt > 0.13, and your project is inside the folder mainprojectfolder, do the following:

1. Make sure the file mainprojectfolder/project/plugins.sbt exists and contains the following line :
   addSbtPlugin("org.scala-js" % "sbt-scalajs" % "0.6.4")
2. Modify the build.sbt file of your top-level project to compile the javascript project when it is built. Add the following lines:

   lazy val js = (project in file("js")).enablePlugins(ScalaJSPlugin)
   scalaVersion in js := "2.11.1" // Or the right version
   addCommandAlias("runWithJS", ";fastOptJS;run")

3. Make sure the definition of the root includes the following aggregation keyword.

   lazy val root = project(...).aggregate(js)

4. Create your scala file(s) at  mainprojectfolder/js/src/main/scala/your/package/myscript.scala with the following content (or something from the tutorial):

5. package your.package
   import scala.scalajs.js.JSApp
   object YourApp extends JSApp {
     def main(): Unit = {
       println("Hello world!")
     }
   }

   Now, after running sbt  inside mainprojectfolder I, you can launch the following  new tasks:
   - fastOptJS  which compiles the new scala file above to javascript
   - runWithJS  which performs the fastOptJS  above plus the run command (you can customize this according to your needs).

   These two tasks both create the following files inside mainprojectfolder/js/target/scala-2.11 :
   - jsproject-fastopt.js: This file needs to be included in the final html.
   - jsproject-fastopt.js.map: This is the source map
   - jsproject-jsdeps.js: This file needs to be included in the final html. It contains all the remaining dependencies.

6. You may be interested to copy the generated javascript files to another folder like mainprojectfolder/public/js. For that, add the following line in the build.sbt file:

   lazy val copyjs = TaskKey[Unit]("copyjs", "Copy javascript files to target directory")
   copyjs := {
     val outDir = baseDirectory.value / "public/js"
     val inDir = baseDirectory.value / "js/target/scala-2.11"
     val files = Seq("js-fastopt.js", "js-fastopt.js.map", "js-jsdeps.js") map { p =>   (inDir / p, outDir / p) }
     IO.copy(files, true)
   }

   and of course update the custom command in the same file:

   addCommandAlias("runWithJS", ";fastOptJS;copyjs;run")

Next: Part 2: Integrating DOM and jQuery

Motivating example

Recently, I came across different errors in Typescript which bothered me. Although I am not an expert in type theory and I love Typescript, let me explain what I think could be made better.

A method returns an IIterable<TreeElement>, where the TreeElement is called a type parameter. The method fromArray is generic and accepts an argument of type T[] and returns an IIterable<T> for any T.
Resharper 9. gives me the following suggestion.

When I accept the suggestion, I get the following new error.

Why is Typescript, like many other languages, unable to perform such correct type checking?

It is because type checking is not just about verifying types. It is really about proving program properties. For that, I believe that engineers can really learn a lot from Academia.

First, let me show that it is possible to correctly type-check the sentence above.

How to solve the type equation

For example, with the first example, it gives the following type equations:

TreeElement[] <: T[]
IIterable<TreeElement> >: IIterable<T>

This means that

1. An array of TreeElements should be viewed as an array of T to pass to the method .fromArray
2. The result, and IITerable<T> should be viewed as an IIterable of TreeElements

A trivial solution exists: T = TreeElement. Is it really the only solution? How to solve it in the general case?

A word about variance and contravariance

The type parameter of an array is nor covariant nor contravariant. This means that:

1. No covariance: Covariance would mean that an array of apple is an array of fruits. If it were, then we could put a cherry in the array of fruits and the original array would not be an array of apple anymore.
   a =  [,,,] is an array of apples.
   An apple is itself a fruit.
   But if we say, to abstract, that a is also an array of fruits, we would put a cherry at the first element.
   a[0] = 
   but then a would be  [,,,] which is NOT an array of apples.
2. No contravariance. Contravariance would mean that an array of fruits is also an array of apples. This is obviously wrong.

Therefore the first equation yields the unique solution T = TreeElement which satisfies the second.

Without the explicit type parameter?

The equations become trickier if we remove the explicit type parameter, as the suggestion tells it. In this case, the array can be an array of anything, let's say an array of U where U is an unknown type variable.

U[] <: T[]
IIterable<TreeElement> >: IIterable<T>

Because there is no way to define covariance and contravariance in Typescript (as of 1.4), the only way IIterable<T> can be a sub-type of IIterable<TreeElement> is that T = TreeElement. Filling this solution to the first, we get:

U[] <: TreeElement[]

where we have the result that U should also be TreeElement for that to work. So this should work as well...

Why does Typescript fails there?

Like many other examples, Typescript does not set up type equations or type constraints. It solves equations in a greedy manneer, without back-tracking. Although this is viable for small programs, this does not scale and it's a shame.

Alternatives to Typescript?

if not developping with Visual Studio, I would recommend modern languages such as Scala, which now can compile to Javascript.

The type checker does this thinking and we can define the following statements without error.

class IIterable[T]
object IIterable { def fromArray[T](input: Array[T]): IIterable[T] = ??? }
def testMethod: IIterable[TreeElement] = IIterable.fromArray(Array())

It will correctly infer the missing type parameter for the array. Besides, thanks to the huge collection available, no need to reinvent the wheel.

But what are types?

Although the question of solving type equations is trivial here, this question is less trivial in general, because if the type system is sound and complete, types can generally encode arbitrary mathematical notions. Even without predicates types, like a simple int {x=> %2=0 x=>x>3} x , we can encode many mathematical problems in types which the compiler will have to solve, and sometimes will fail.

This is why it's hard.

Conclusion: Please add a modern type checker into Typescript.

Here is what is currently missing in Typescript 1.4

• Support for abstract classes.
• Support for type constraints, like mentionned above.
• A solver for type constraints.
• Support for F-bounded polymorphism.
• Support for traits or mixins.

It's worth it for many reasons:

• Types help refactoring
• Types provide guarantees about your program
• Types structure any home-made library and make component reusable
• Types provide clever code completion
• ...

Useful references:

- A Core Calculus for Scala Type Checking, 2006, Vincent Cremet, François Garillot , Sergueï Lenglet, Martin Odersky

I recently came accross a nice easter egg provided by the Google Android Team - as they often do it - on my galaxy tab s1.
By visiting the Settings page and typing multiple times on the version number (4.0.2), it opens an android in a sandwich.

By taping around, it gets bigger. Finally, it becomes like a kind of Nyan cat of droids without sound.

This was so funny I decided to try the same on my phone which runs on JellyBean.

Note how the jelly seems happy to draw attention from important men. The background picture is mine, not Android's of course.

Enjoy !