47:09Yes.
47:09So data is interesting because
It's so easy to collect
47:14enormous amounts of usage data.
47:17Was this feature used?
47:18Yes or no.
47:19How many times per day was it used?
47:21Was it used this week?
47:22You know, was it used in the
first month of the person?
47:25Doing it or only after a month to,
you know, like you can, you can slice
47:28things a million different ways.
47:30so in products past, I have often said,
well, I don't know what's important.
47:37So I'm just going to collect a bunch
of data and I'll figure it out later.
47:39And then later comes around and
I have a huge pile of data that
47:43I don't know how to look at.
47:45And it's just overwhelming.
47:46So I wanted a completely different path
this time on, this was December, 2023.
47:53So a handful of months after taking over
Muse, I'd already done a lot on bug fixes.
47:59It was starting to kind of get, okay,
new users are happy, existing users are
48:03happy, the fires small as they were,
they've been put out for the new release.
48:08Let's look at the data and
figure out what's important.
48:10What do I need to look at?
48:12so in times past, I've had way too
much data and I didn't know how
48:17to pull out the answers from it.
48:19And so this time with Muse, I've been
very purposeful about saying, what are
48:23the important questions I need answered?
48:25Let me clarify to myself.
48:27What do I actually care about?
48:30What is the most important
thing that I need?
48:32And then let me go collect data
specifically to answer this question.
48:36And that's it.
48:37And maybe that data could be
used for other questions too.
48:39And there's all sorts of different
stuff there, but I'm very purposefully
48:43limiting what I look at to only
the questions I know matter.
48:50And so the biggest question, that I had
initially going into it was that customer
48:55funnel, how many people hit, hit the App
Store page, how many people download,
48:59how many people log in, how many people.
49:02subscribe, and then there's kind of a,
a middle one, which I call activation.
49:06So between logging in and
subscribing, it's, are they
49:10getting value out of from Muse?
49:12Like, have they done something that
they've at least played with it enough
49:17that yeah, it seems to be, they understand
what they're saying yes or no to.
49:21So the first thing I did is I
downloaded, We, we don't use
49:26generally any third party trackers.
49:28So all of the data we have about
user behavior is on the Muse server
49:33and is not shared with anyone else.
49:35So it's not used for advertising or
for, you know, various other things.
49:40that's been a very important piece.
49:42And so I've been able to look at that.
49:44Kind of feature usage data.
49:46We don't collect any data in terms of
what are you physically typing into Muse?
49:51It's all about, did you use note cards?
49:53Did you use links?
49:54Did you use boards?
49:55That sort of stuff, right?
49:56Do you pull this out out of the sync
data or is that a separate thing that's
50:01completely separate from the sync?
50:03It is completely separate.
50:05And so, and there are no circumstance
in my poking around inside of sync data.
50:11that is a hundred percent kind of.
50:13Private tucked away.
50:15And then there's a separate piece
that is just product usage data.
50:20And so that, and that collects
none of the personal information
50:24that you're putting into Muse.
50:25It's only collecting.
50:27You know, sort of, did you click this
button or not kinds of data so the
50:31first thing I did is I downloaded,
did people use this feature?
50:35Yes or no, across 30 different features,
30 different, 40 different things.
50:41And then did this person subscribe or not?
50:43And I gave me a giant table
of data that I looked into and
50:47said, okay, which of these.
50:50features using which of these features
is or is not correlated with subscribing
50:56and I narrowed it down to, I think,
six and so if people use all six of
51:01these features, then they are more
likely to subscribe than not and.
51:07What that means to me is, it's obviously
not just, okay, great, let me go force
51:13everyone to do these six things and then
clearly they're going to subscribe more.
51:17No, what it means is that, okay,
doing these six things gives them a
51:21real good feeling for what Muse is.
51:23And once they have a good feeling for
what Muse is, those kinds of people are
51:26going to more often than not subscribe.
51:29So I have that activation.
51:31That's what I call activation.
51:32so the report that I run connects
to, app figures, which connects to
51:36the App Store for App Store metrics.
51:38I can also connect to the App Store
directly because there are sometimes
51:42information that I want to get
kind of the raw data for instead
51:45of app figures, aggregated data.
51:47I connect to the Muse server to get, more
detailed analytics about subscription and
51:52about activation and things like that.
51:54and I connect to the, we use
Fathom for website analytics.
51:59So it is a very privacy conscious
website analytics tracker.
52:04And so that gives me number
of visits, number of click
52:06throughs, things like that.
52:08so I pull all this data from three
or four or five different sources.
52:11And then together that gives me full
visibility from number who see the
52:17website, click through the App Store,
download link all the way down.
52:21And so once I have that data,
that's when I can say, okay, let
52:25me look at new user onboarding.
52:27What happens if I provide this kind
of video, or if I provide this kind of
52:33tutorial, or if I change this kind of
thing, is that better or worse for this
52:39single step from download to activation?
52:42Not even caring how it affects
subscriptions or anything else, but
52:45like, can I just change this metric?
52:48and so the times I've done this
over the past year, year and a half
52:53have been for onboarding, of course.
52:55So the first tutorials
that people can get.
52:58Also, Setapp has helped because Setapp
takes out the subscription altogether.
53:05And so then that very last
step from download to log in to
53:09activation to subscription p user.
53:12The only thing I need to care
about is download to log in to
53:16activation once they're using these
consistently, then that's when
53:20Setapp recurring revenue comes in.
53:22so that was important on the Setapp side.
53:24On the App Store side, I implemented,
sign in with Apple because Muse requires
53:31an account, for the sync server.
53:34That means the first time download
experience, people load up Muse and they
53:38see, hi, give me your email address.
53:40Muse is very conscious more than I
think almost any other company I've
53:45seen or worked with about privacy.
53:47But when the first time user experiences.
53:50Hey, buddy, give me your email address.
53:52It doesn't, it doesn't inspire confidence.
53:54and so I implemented sign in with
Apple and then that lets people
53:57say, okay, let me use that.
53:59I can choose a private email address.
54:01I can maintain my privacy, but still
kind of create the account that allows
54:06for them use sync service to work.
54:09So that helps the download to login
step of that entire funnel flow.
54:15And so it's been rewarding to.
54:17focus on very specific places in that
funnel and say, okay, this piece right
54:23here, right after the download, what
kind of context does that person have?
54:27What do they need?
54:28What would be helpful?
54:29maybe new images in
the App Store or maybe.
54:32better tutorials on the website, or maybe,
you know, fill in the blank, but how
54:37can I get this from 92 percent to 96%?
54:42And then in theory, that will also have
downstream effects at the bottom of the
54:45funnel, but if for whatever piece that
I'm looking at, that is the biggest.
54:50problem that has been very helpful
from a prioritization standpoint.
54:55And that has been very helpful, to keep
me focused because they're, you know,
55:00like I've said before, there's too
many things for me to work on that I
55:04have time in my life to physically do.
55:07And so when I am building, it can
be motivating and really helpful
55:11for me to say, Okay, Adam, remember,
you're focused on helping this
55:15person at this step in their journey.
55:17they would love to use Muse, but
they can't because they're stuck.
55:21And so you're going to help them.
55:22How how can you help this
kind of person get unstuck?
55:25and see what Muse is so that they can
decide whether it's a good fit for their
55:28life or not and for their workflow or not.
55:30and so that's been very helpful to
collect very specific, and still
55:36privacy preserving data that helped
me make decisions in terms of that.
55:41That flow, there's a handful of
other statistics I look at in terms
55:44of like App Store revenue or Setapp
revenue, subscription counts,
55:49cancellations, those sorts of things.
55:51But broadly speaking, that funnel
data has been the most important and
55:55for prioritizing my, my work and in
the world of data, it's a very small
56:00piece, compared to the data pile.
56:03I've seen at other companies or
in previous things, it's it's
56:07really helped keep me focused.
56:09In terms of Muse being a local-first
app, as opposed to being like a more
56:14traditional, cloud based SaaS app.
56:18Is there anything that you thought
about different when it comes to,
56:23getting better insights through
data into how users are using it?
56:27So, there's this interesting balance
between, uh, local-first really tries to
56:31preserve the privacy, a user and you with
the best intentions of like, Building this
56:38app for the people who you want to serve.
56:41And yet you need a little
bit of visibility into this.
56:44Have you thought about this for Muse
differently than for previous apps?
56:49And did you build the analytics
stack from a technological
56:52perspective in any different way
than you've built previous ones?
56:57Yeah.
56:57So when I, joined Muse, in 2020, the
analytics stack that's still being
57:02used was built already and that was,
implemented entirely on the Muse server.
57:09So that way, none of the analytics
data went to a third party.
57:13It kind of stayed within Muse.
57:15And so that was very helpful.
57:16And then, like I mentioned before,
that analytics data that we collect is
57:22entirely separate from the actual synced
data of a person's library in Muse.
57:30Is there still like the same
sort of identity behind it or
57:34how does, user privacy preserving
look like at that point?
57:38Do you, for example, like have something
that is, identifying a user, but you
57:43hash it so you can't like, correlate it
anymore or, how are you going about that?
57:49Yeah, so it does use the same user ID.
57:54And so I can see, which is
helpful for our support tickets.
57:57And so when a support ticket comes in,
I can see, obviously, when the person
58:02signed up, if they're subscribed or not.
58:05And I can also see, which devices they
have synced to the sync server and how
58:11recently those devices were connected.
58:13Because far and away one of the
most common support requests I get
58:18is Usually a one line email that
says: Hey, sync, is it working?
58:21Or, Hey, there's a problem with my iPhone.
58:25Uh, how can I fix it?
58:27And so I can immediately look
and say, okay, I don't see an
58:30iPhone on their account, clearly
it's not connected correctly.
58:34And so that helps me reply.
58:36but that, that is kind of the
only connection is that user ID.
58:39So I do see.
58:41User behavior, and then there's a separate
bucket that has all the user synced data.
58:46But the most important guiding principle
through the entire life of Muse has
58:51always been, the user's synced data,
their library data is off limits.
58:58It, there's just, it's
just never looked at.
59:01It's never looked at by a human and
it's never looked at by a robot either.
59:06Like we don't run analytics on it.
59:08We don't run scripts to see
how things do like it is.
59:13It is its own little box in
the closet that is not touched.
59:17And then that way, the only data that
we see that is used for analytics
59:22is, the feature usage data that we
specifically send, that does not
59:27contain any of the actual library data.
59:30None of the text, none of the ink,
none of the boars, none of the content,
59:33none of that kind of stuff lands there.
59:35It's just, oh, they made a board card.
59:37Okay, great.
59:39I need to know if people make
board cards or not, because if
59:41they don't, what are they doing?
59:42Because Muse is based around
boards and whiteboards.
59:45Yeah, I think it's this interesting
balance where with local-first, we
59:50obviously want to move beyond the
status quo of how software is being
59:54built traditionally yet, or in terms
of how software is deployed and
1:00:00architected in a way traditionally,
but yet a lot of the more traditional.
1:00:05Product management learning still apply.
1:00:08Like we still don't want to fly blind.
1:00:11We still need to understand what
the users are doing, et cetera.
1:00:14So there is a slight tension there
between still like knowing how are
1:00:20our users successful with the app?
1:00:21Are they struggling?
1:00:23Where are they falling off?
1:00:24And yet, The, that the user's private
data is sacred and you don't touch it yet.
1:00:29You don't even have a way to look
into it as it's encrypted, et cetera.
1:00:34So I'm curious, like what will the
ideal analytics stack for local-first
1:00:39apps, maybe look like in the coming
years to have some intuitions or some
1:00:44wishes for like, this is what the
ideal stack there would look like.