Lucid dreaming app revolves around two processes: collect data and analyze data. To help facilitate analysis, the tool automatically plots the data for you. Here’s an explanation on how to work with the graph viewer activity.
Basic sleep graph
Axis and series
- Sleep score – Values less than 1 indicate sleep, otherwise awake. This is the red line
- Sleep episodes – value of 0 indicates awake, 10 falling asleep and 20 asleep. This is the blue line
- Sleep duration rises with each minute you are asleep. This may be useful for estimating periods of sleep interruption. Green line
- Reminder played – shows the minute when a reminder was played. This info may be helpful in gauging the effectiveness of the app. Purple X above the blue line
- Awake, Dream, Lucid Dream, User Event – indicates minutes when the you interacted with the screen. These are shown as little symbols in the upper portion of the graph
Symbols are color coded and shown in the legend. Here’s what they stand for:
- X = reminder played that minute.
- O stands for both awake and no dream events
- Triangle stands for regular dreams
- Square stands for lucid dreams
- Diamond stands for other events and screen interactions
Sleep Statistics
Graphs include a statistical summary of that night’s sleep. Here’s the overview of some important metrics:
- Sleep onset latency – time it takes to reach 5 consecutive minutes of sleep
- Sleep score – a user is considered asleep if the sleep score is less than 1 on 5 consecutive minutes
- Number of awakenings – number of times the app detected significant movement and think you were awake
- Number of user events – number of minutes when you interacted with the screen (max 1/minute)
- Longest sleep episode – longest period with sleep score of less than 1
Analyzing the data
Lucid Dreaming App produces a lot of data, and there’s quite a lot that may be learned. There are 2 modes of analysis: individual graph and History.
To view individual graphs, open the app to see two androids in a dream bubble. Press the menu key on your phone to open the menu bar. From the menu bar, select “View Data”>”Load Graph”.
Lucid Dreaming App typical night's data.
Above is what the typical night of data may look like. So you got fancy graphs… Now what? First thing you should look at is the duration of your sleep episodes. These are shown as the raised blue line. The line is raised for each minute you were in deep, motionless sleep.
As you might’ve guessed, the blue and green line show the same data just in different formats. The green line shows total minutes asleep and raises as you accumulate more sleep. By analyzing the green line, I may see:
- When I fell asleep (~2:30 AM) – this is when the line starts rising
- Identify periods of restless sleep – this is when the line is flat (~7:00-7:30 AM)
- Find out the total sleep duration (Y axis) vs Time (X Axis) (200 min at 6:40, up to 290 min at 7:20)
By itself, the green line does not tell us much, unless we look at the history of data. (I’m getting ahead of myself here, but here’s a comparison of the night above with an night that I had earlier in the week:
An insomnia night vs regular night
By recording sleep data every night, you will be able to compare your sleep graphs over 1 week period. This will help you determine the best time to go to bed, as well as validate any pre-bed rituals that you might do: for example comparing quiet meditation (before blue line) to watching exciting online podcasts (red line).
XYZ Activity and the Sleep Score
When your phone is placed on your bed and the Lucid Dreaming App is running, it automatically collects and filters accelerometer sensor data. Based on this, it produces an activity level very close to 0 when the app is laying flat, while registering activity spikes produced by movement and mattress shaking:
Sleep score is calculated from filtered activity data
The Android Lucid Dreaming App uses Actigraphy data to automatically score sleep, according to Cole sleep scoring algorithm. The algorithm looks at a the activity data for the past 4 minutes, the current minute and the following 2 minutes to score sleep.
Score below one indicates sleep, score above 1 indicates waking state. Sleep score rises in proportion to the movement.
To better view activity and sleep score for your phone, you may change the Y Axis resolution for both sleep score and activity level. To do so: press menu button > preferences>scroll down to display options> scroll down to sleep score Y Max and Activity count Y Max. Set desired values and reload the graph.
Changing the Y Max allows you to either clip data that is too high or zoom in on the data you really want to see
Notice that changing the Y Max for Activity count from 2500 to 1000 makes it a lot more visible on small screen! I also changed the sleep score from 35 to 15.
Analyzing sleep score and activity level allows to determine the following:
- Activity level above a certain threshold (2000 for my mattress) indicates getting out of bed
- Activity level of ~500 means writing in a dream journal or getting a drink of water, or rolling over in bed
- Sustained activity level (like from 7:00 to 7:30 may indicate restless sleep) Use “After movement” Smart Timer event to deliver reminders during such episodes, as they may indicate very light sleep.
- A jittery sleep score graph means you need to calibrate your accelerometer or restart the phone before running the app
- A jittery activity graph means your accelerometer works in a way that I did not anticipate
(I have only 1 of over 370 Android phones out there!)
Analyzing User Events
Android Lucid Dreaming App uses the phone’s touch screen to capture user events in form of on screen gestures. Gestures are drawn by touching the screen and dragging the finger, drawing a desired shape with the yellow line that follows your finger.
Main gesture types:
- Normal Dream – triangle
- Lucid Dream – square
- Awake – circle
- No Dream – circle
- User Event (any gesture that you define) – diamond
- WILD Timer – not shown
- Math – not shown
Of particular interest are the normal dream gestures, especially if entered immediately after waking up from a dream. They indicate that an REM episode was experienced right before the awakening.
User events reflected on the graph. Triangles are dreams, Xs are reminders, Diamonds are misc events
On the graph above, we can see that 4 reminders have been delivered by the app that night. They correspond to the Smart Timer events that I’ve created. 3 Dreams were reported (barely visible yellow markers) at 5:30AM, 7:30 AM and 9:10 AM. We can see that one reminder – at 5:25 occured before the dream was reported. This is a good indication that the reminder might’ve been integrated into the dream’s content, or that it caused an awakening from a dream.
Entering dreams over time is very important, as it allows you to analyze them using History feature later:
Notice how triangles (dreams) over 5 days cluster around specific times (all times are measured from time in bed). You may configure smart timer to deliver reminders around those times.
[Add history analysis section]
Audio Level Analysis
An experimental feature that is coming in V0.7.1 is audio level analysis. No sound is recorded and stored, but rather short audio recordings are being continuously analyzed and discarded. By continuously sampling data in 1 second intervals at 8Khz, it is possible to determine the sound intensity which takes place during sleep:
Audio level Kurtosis (peakedness) - blue line and motion activity (yellow line)
High Kurtosis indicates that a data set has a few rare, high outlying values.
If we examine the data above, we notice that audio level indeed has high values around periods of activity (ex: bed squeaking, covers rustling), but there are additional activity immediately before the dream is reported. This may mean talking in sleep, grunting, snoring or other voice related cues which are not intense enough to be reflected on the motion chart. On the other hand, these can also be sounds coming from the outside.
Audio level display will likely change in the future. I envision it being useful for:
- Noise analysis versus sleep quality. Notice how noise picks up at around 5:20 Am as the highway outside starts to pick up traffic.
- Voice cues analysis – snoring, speaking, etc
- Disturbance analysis – another person was in the kitchen from 6:40 to 7:20AM, preparing for work.
- Potentially delivering reminders on sound during REM episodes
Section on working with history is coming soon!
Loading ...
I can’t get my graph to load.:(
Sindy, the app comes with a sample graph. Can you load that one, or is it just the one from your own nights?
Downloaded the app yesterday and first use was last night. This is a great app. I am more creative than technical so it is taking me a bit to understand all the information – but I am amazed at the detail captured during the night. I look forward to learning more on usage and details each night. Thanks for a great job in creating this app.
How do you delete information? Is that a function you’ve not yet added?
Hi Zoneja,
The data for the app is stored on your SD card in /application data/lucid dreaming app/graph data . You can manipulate content of that folder in any phone file manager. Generally full night’s graph data is over 15kb in length and you probably want to delete test files and such, which are less than that.
I hope this helped!
So it seems like the reminders are not working on my app. It says it cannot find the lucid.mp3…any ideas?
Hey Meredith,
The app comes without any audio files, because they are large and will not work for everyone. This is why you need to create a reminder yourself and point the app to it in order to use it. Try using the “Get/Start MP3 Recorder” from menu>preferences>set up sound reminders>get/start Mp3 recorder.
I have used this app four times and three out of the four times it has told me that I am not sleeping at all and one night is said I selpt for four minutes.
I have calibrated the app each time before I’ve gone to bed and am not really sure what i am doing wrong…? Any suggestons? I am not getting any symbols and my sleep score is between 5 and 10.
And if the app isn’t recording sleep… it won’t play the reminders?
I really hope to use this app, but it just doesn’t seem to working right for me.
Thank you for the feedback!
There are indeed some phone which work differently. There are about 400 models of Android phones right now, and I only have 1 to test the app on. It sounds like your phone’s accelerometer is working differently than mine, and the calibration does not work as intended. I can try to fix this, within the next couple releases, although the workaround may add extra complexity to the app.
elizabeth,
What model phone do you have and what version of Android does it have on it. 3 out of 5 nights I experience the same situation. Though I wonder if perhaps it is from operator error (me) perhaps there is to much ambient noise in the room or my wife contributes to additional noise and movement.
Alex, You are a genius. We understand that your are a sole developer. Props to you buddy this is a difficult role especially with the iPhone app that you released. If i can help in any way with this bug report (if it is a bug at all and not operator errors) I am using the LG VM670 Android Version 2.2.1 Kernel version 2.6.32.9
There are over 400 android devices out there, and I found (painfully so) that some of them do not work as my device. Particularly their accelerometer updates are not delivered as the app expects. Chances are I will never touch most of those devices, and will not be able to resolve the issue. There’s a list of phones on which there were confirmed issues and phones which worked. This is why there are 2 calibration modes and you get the option of adjusting R. There are no such issues on an iPhone, there’s 1 iphone and I can debug issues much quicker.
The point is that the Android may not be the best platform for this kind of an app due to tremendous difference in hardware and software implementations that the phone manufacturers use.
That’s understandable. For reference, I am using the Samsung Infuse which was released last month.
Even though it’s not working as well as I had hoped, it’s a pretty incredible app. Thanks for creating it!
You are welcome ! Sometimes it’s tough being a single developer and listening to so many interested users at once. I’ll do everything in my power to make this app successful for lucid dream induction!
I can’t get the graphs to load on my phone. Do I have to download it to my computer?
I just downloaded and tried this app last night. Not being technical enough to fully understand what’s going on, I wasn’t sure what to do with the one night’s worth of recorded data or how to interpret it.
My only reference to the graph that was on my phone from last night are the charts shown on this page. I have a Samsung Droid Charge. This may be why the graph is very “jittery” compared to the charts here. FYI: I tried changing the graph y-axis display settings several times, but nothing seems to effect the chart when I reload it.
One thing I wonder. Is it possible to upload the collected data files to view them via my desktop computer? My Charge is nice and all, but these rich graphs deserve to be viewed via a larger display.
Another question: In addition to the graph, just after I woke, in addition to the graph I recalled also seeing a info-rich data table, presumably the Sleep-Score_Log? While I can view the text file, is there an online reader that will translate and replicate that table I saw on my phone?
I suspect it may be quite a while before I can reap some lucid dreaming benefits from your app, but at the moment I don’t mind. I love this kind of stuff and I appreciate the work you must have invested in providing this app.
Desiree
Hi Desiree,
If the graph is jittery, you may need to re-calibrate the app and find the correct tuning R parameter as described here: http://luciddreamingapp.com/help-how-to/calibrate-the-app/
Once you got a few days of data, you can view those graphs superimposed upon each other by viewing “history”. If your app is tuned, it will make sense – you will see peaks on the graph appearing at similar times. This is when you are in a light sleep. Read this post to learn more on how to deal with history: http://luciddreamingapp.com/help-how-to/detect-sleep-cycles/
As for the text data that is scrolling on the screen, it is used for debug/troubleshooting purposes mostly.
Thanks, Alexander.
I’ve tried numerous extended calibrations over the past two weeks, without much success at eliminating the “jitters”. Then I noticed one thing – the various values I put into xR, yR and zR for calibration tweaking usually are not saved.
I’m not sure why or why sometimes they’re saved and sometimes not.
Thank you for the bug report! I’ll look into the possible causes of values not being saved
Hi!
I’m trying to analize the data by my own, and I’m trying to interpret the json data from gzip file.
Is it explained in any section of the webpage?
I’d like to know how is the date encoded, is not a simple unix time?
Thanks!
Jason,
Once you unzip the gzip with a program like 7zip, it is just a text file in JSON format. You will be able to load this file from a programming language (for example Java with google GSON ) and then iterate over the sub arrays.
The timestamp next to JSON values is in a format from 12:00 to 24:00 to 12:00, in other words it encompasses 24 hours since noon one day to the noon of other day. This lets data to be plotted in a way that compares time in bed, awakening time, etc.
To get the data’s UNIX timestamp, you need to add the “timeStampStart” and “timeZoneOffset” to the timestamp data for each sleep minute. This gets you the UNIX timestamp of each minute. If you do not add timeZoneOffset, you get a timestamp pretending to be in the UTC time.
A few questions:
1. Is it possible to export the graphs and history to an excel file so I can see it better? I get the gist of it on the phone but it’d be so great to see them enlarged and also be able to email or post them.
2. I listen to binaural beats to try and induce a lucid dream. I wear headphones. Is it possible to make your app take priority? Like the lucid dream track is playing, but when it’s time for the reminder to come through it will interrupt and then go right back to the binaural track?
And thank you SO MUCH for this app! I was thinking of switching back to an iPhone because they had the best sleep cycle app… until now! I absolutely love what you’ve done. It took me a few weeks to really get used to it and set up the proper reminders, but after the learning curve it works flawlessly. Thanks again!! You rock!!!
Hi April,
At the moment the app can only export large pictures of the graphs. There’s a running log of data being written for each night, it is in the “logs” folder. From that, you might be able to graph stuff in excel.
I can’t load my graph… I don’t know if I’m just missing it, but I can’t find a “load graph” button anywhere
Load graph button is available if you start the app and tap the menu button (soft button or hard button) on your android device. This brings up a menu ribbon from the bottom of the screen. Once on that screen, hit the menu button again to select the graph file.
Hi mr stone. i tried many apps before but notng happened. This app is very useful so my questio
n. When i was a dream i had made a gesture so i know that i was in dream last night. what should i do to make my reminder to this time. wgen i had done this. it brings me to a lucid dream right?
greetz from germany
Check the “configuring smart timer” article. You need to know when your dream happened (look at the graph) and then set the smart timer for that time
I’m sure you know this by now, but despite your app’s brilliance it doesn’t work for the average user. The graphs are inaccessible. The view graph button just pulls up a list of instructions about charting and viewing data. I do hope you will make revisions as this is one of the most interesting apps I have ever seen. Kudos and keep working on i
Hi Kian,
I’m still working on the app. The graphs are a work in progress, I’ll publish an excel spreadsheet that will plot data better than any mobile phone can.
One thing that can help you manage complexity is that the app is there for the long run. If you are into lucid dreaming, I expect that you would keep using the app day after day. So some of the things that look complex for the average user would save you a ton of time after a couple weeks of use.
Additional reason for complexity is that the app is your gateway into lucid dreaming. Lucid Dreaming requires different kind of thought. I’m hoping that the hypnotic interface along with the (un)intuitive controls would help you get a sense of this kind of “aha, this is how it works” feeling that is the key to becoming lucid in a dream.
Access the graph through the options menu while viewing the instructions page. So it’s options >> view data >> options >> graph. I agree that this needs to by more forward and intuitive, but it’s not broken.
I used this app for the first time last night and am experiencing some troubles with the data. To start, I will say it was a rather restless night thanks to a huge difference in sleep cycles due to it being Christmas. I was only awake for 10 hours sleeping with the app running.
That side, my graph seems to have some errors. It indicates that I have achieved 0 hours of sleep with an expected 0 dreams. There is data for the accelerometer and audio capture, but it simply hasn’t recognized any sleep episodes.
It’s there a place where I can submit my data and device specs to have this problem looked into? I noticed some others have a similar problem.
Device: Samsung Galaxy S 2 Epic 4G Touch; Stock (not rooted)
The app seems to work fine on my Droid x. The only complaint i have is the phones buttons stay lit all night. Do you know if it is just me or have you heard of this before?
Thank you
Craig
That’s a common complaint with the Droid device. I haven’t been able to figure out how to dim them.
I’m using this on an ARCHOS Android tablet, and am mainly interested in the WILD timer, although the graphs are really interesting too. The WILD timer works great, except I wonder if there’s a way to configure multiple timers for different base sequences, I sometimes set the timer when I go to sleep for five hours hence, and sometimes wake up and then I want the timer to start from 00.00 if that makes sense.
Does it always read from the same (configurable) timer file, or is there a way to have two or more timer files to choose from?
Nice work by the way. Liking this a lot, except for the odd time that the sound reminder comes through really loud and distorted, usually when a music app had been loaded earlier or is still in memory.
Excellent app. I would gladly pay for it. The fact that you did this on your own (without a team of developers) is even more impressive. I look forward to you improving the getting started/tutorial sections.
Hi. Thank you so much for this app. I am using on a Droid Incredible 2 and for the most part everything seems to work great. I may be using it more to track sleep patterns to try and fix issues I have with not getting good sleep and having a very hard time waking up.
I only have one minor problem right now. The graph confirmed for my first night that it did issue 3 reminder events that corresponded with light flashes I vaguely remember. I have reminders in the settings turned off. Do light reminders have to be configured seperately? I didn’t see a control for them. I am starting data collection for now before trying the lucid dreaming aspect.
Thanks again. Great work.
If you go into the preferences, there’s an option to configure smart timer. Seems like you may ned to manually go through events and turn off the light reminder.
The first time I used this app(this morning) I think I reached a WILD, can someon please identify if it truly was? First everything got tingly, then a sort of buzzing noise came, I was conciously thinking and could still see but it felt like my body was paralyzed. Anyways I got into the dream but was quickly pulled out. I was trying to find one of my friends, but the second I did the dream ended. All information would be appreciated
This could be sleep paralysis or WILD onset. A good indication of WILD is that you can see with your closed eyes while your body is buzzing.
My sleep sleep episode looks nothing like this. This shows the episode at discrete levels and mine kind of steps down a unit every 5 minutes or so.. is that right?
I have the same problem…
Any advice?
mine is also not functioning as demonstrated. my sleep episode graphs are all over the place. when I view the history the hypnogram is there, then mine are on the top of the graph or a line at the bottom of the graph which makes the whole thing unreadable. my xyz graphs also don’t start at 0 but above maybe at 5 or 10. I’ve calibrated it a couple of times. I have a galaxy s3
I have the same issue, after I have calibrated a few times, made sure that everything is correct in the settings, I get a staircase style sleep episode graph with my sleep episode graph starting at about 16 and then it goes down by one unit every 6 minutes or so then when i have an “awakening” it goes back to the top and starts again and doesn’t show me clearly if i’m in any type of sleep stage.
What am i supposed to callibrate to fix this? I wondered could it be that I’ve calibrated to use different LowR values (0.0008,0.00008,0.002) should I only be using the default values multiplied by 10?
I havn’t had a perfect calibrated graph yet i’ve been trying for a few weeks to fix it.
i had use this app several times but i didn’t get any dream at all. i am a dreamer, i might get 2-4 dreams every night. but none of them become lucid dream.
I think I have the same problem of blake and Simon. My data looks strange:
http://i46.tinypic.com/103v5f4.png
I’m on a Samsung Galaxy Ace. I can provide the raw data too, if it’s useful.
Thanks!
Guys, here’s how to interpret the sloped graph:
When you are laying still in bed, the graph would slope down in steps. So you would be looking at at | triangle. The graph would keep sloping down as long as you lay still. Once you stop laying still, the graph would stop sloping down, and would jump straight up.
I will analyze the graph you linked as an example:
You went to sleep close to midnight, fell asleep within ~15 minutes, after which you laid still until about 2:00, at which point you moved. So from 0:15 until 2:00 is your sleep episode.
After 2:00, you started to lay still once again, but this time for about 45 minutes, then you started moving once again.
So the blue line that was sleep episodes now shows sleep episodes as downward sloped triangles.
The reason why I made it that way is that there’s an algorithm that I wanted to run on the device, but did not implement it.
Why does my graph always show I me having 100% non restful sleep. I know that can be. Please help.