Wednesday, June 17, 2015

Decoding wireless weather sensor data

Being on the hunt for the cheapest wireless temperature and humidity sensors led me to FreeTec NC-7345-675 - sold by pearl.de at the fair price of 10E, though I might have paid even less for them.
The requirements were: multi-channel, have at least 0.5C accuracy and provide humidity readout. Rolling your own can easily cost more than that.
The sensors pair with this weather station.

The goal is to store all the data on an SBC and provide a nice HTML frontend with long-term statistics and heating automation.

EDIT: blogger removed without warning all the newlines from my code, I had to redo some of those

Hardware

The first step is to open up the sensors and see what we are dealing with.


With the main board detached from the zebra display connector, we can see the coil antenna at the top and the wires for the humidity sensor running at the bottom right.
Judging by the traces going from the SW-CH part directly into the chip it seems there is no way to force additional channels.




On the other side, the top part is the 433Mhz transmitter. Behind this small board there's also a crystal oscillator. Beneath it, to the left, is the temperature sensor, and, to the right, the watch crystal.
The sliding switch at the bottom right allows selecting between the three possible channels.
I have no idea what the part at the bottom right does - it looks like a switch but it's actually a trimmer resistor with the top cut off. It might adjust how often the sensor sends data or the power.


Encoding scheme

I've connected the DATA pin from the TX1 board directly to the oscilloscope so I can get clean readings, without any radio receiver noise.
It seems that with each update the sensor sends 9 packets with a length of 36 bits each.
In the screenshot below I've marked the 0s and 1s to aid in decoding.


Measuring the signal yields that the encoding scheme is: 480us time unit = T. Each 0 bit is represented by T+2T. Each bit 1 is T+4T. The spacing between packets is ~8T (3900us):


Software tools

Next, I took the rc-switch library for Arduino and made some heavy modifications to it: instead of keeping all the timings in an array I moved them into a circular buffer.
The library had to be updated to support peekInt:
int ByteBuffer::peekInt(unsigned int index){
 int ret;
 byte *pointer = (byte *)&ret;
 pointer[1] = peek(index);
 pointer[0] = peek(index+1);
 return ret;
}
This allowed me to continuously capture packets while the decoder was still running, and process them in any order without having to worry about array indexes.

Then a new decoding protocol was implemented with the main purpose of writing out timings on the serial console:


The first line is the serial output, the second one is where I marked the potential bits and the third line is the data.
This can be done even with the current rc-switch library, just add the following code after all the receiveProtocolX functions have returned false:
for (int i = 0; i<changeCount ; i++) {
  Serial.print(timings[i]);
  Serial.print(" ");
}
Serial.println();
However this is running inside the interrupt 'thread' so it cannot be left there. It's time to write a custom decoder that will discard a few of the first few bits and decode the 0s and 1s.
bool RCSwitch::receiveProtocol5(unsigned int changeCount){
    unsigned long code = 0;
    unsigned long delay = 480;
    unsigned long delayTolerance = delay * RCSwitch::nReceiveTolerance * 0.01; 
    unsigned int size = buffer.getSize();

    if (buffer.getSize()<(26*2)){      // clearing the buffer will be taken care of
      return false;
    }
    Serial.print("entering with changecount:");
    Serial.println(changeCount);    // remove some bits

    Serial.print(buffer.getInt());
    Serial.print(" ");
    Serial.print(buffer.getInt());
    Serial.print(" ");
    Serial.print(buffer.getInt());
    Serial.print(" ");
    Serial.print(buffer.getInt());
    Serial.print(" ");
    Serial.print(buffer.getInt());
    Serial.print(" ");
    Serial.print(buffer.getInt());
    Serial.println(" ");

    unsigned int firstSignal;
    unsigned int secondSignal;

    while(buffer.getSize()>4){
      firstSignal = buffer.getInt();
      secondSignal = buffer.getInt();
      if (THRESHOLD(firstSignal, delay, delayTolerance) && THRESHOLD(secondSignal, delay*2, delayTolerance)) {
        code = code << 1;
        Serial.print("0");
      } else if (THRESHOLD(firstSignal, delay, delayTolerance) && THRESHOLD(secondSignal, delay*4, delayTolerance)) {
        code+=1;
        code = code << 1;
        Serial.print("1");
      } else {        // Failed
        buffer.clear();
        code = 1;
        Serial.println("resetting");
        Serial.println(firstSignal);
        Serial.println(secondSignal);
      }
    }
    Serial.println();
    code = code >> 1;

    if (changeCount > 52) {    // ignore < X bit values => noise
      RCSwitch::nReceivedValue = code;
      RCSwitch::nReceivedBitlength = (changeCount - changeCount % 2) / 2 - 12;
      RCSwitch::nReceivedDelay = delay;
      RCSwitch::nReceivedProtocol = 5;
    }
    if (code == 0){
        RCSwitch::nReceivedValue = NULL;
        return false;
    }else if (code != 0){
      return true;
    }
}
The output from a code similar to the one above, amended manually with the values that were being displayed by the sensor and the chosen channel:
29.6 38% ch3
Received 1960 / 480bit Protocol: 480
entering with changecount:73
484 972 01110001010000100101000111100100110
Received 168988454 / 24bit Protocol: 5
entering with changecount:69
1016 1288 110001010000100101000111100100110
Received 168988454 / 22bit Protocol: 5
entering with changecount:69
936 1368 110001010000100101000111100100110
Received 168988454 / 22bit Protocol: 5
entering with changecount:69
880 1420 110001010000100101000111100100110
Received 168988454 / 22bit Protocol: 5
*********************************************
29.6 37% ch3
Received 492 / 1952bit Protocol: 988
Received 996 / 468bit Protocol: 488
entering with changecount:73
480 988 01110001010000100101000111100100101
Received 168988453 / 24bit Protocol: 5
entering with changecount:69
1016 1288 110001010000100101000111100100101
Received 168988453 / 22bit Protocol: 5
Received 980 / 484bit Protocol: 484
entering with changecount:73
476 992 01110001010000100101000111100100101
Received 168988453 / 24bit Protocol: 5
*********************************************
29.7 42% ch3
Received 1968 / 468bit Protocol: 488
entering with changecount:73
480 988 01110001010000100101001111100101010
Received 168992554 / 24bit Protocol: 5
*********************************************
27.4 49% ch2
Received 1000 / 480bit Protocol: 480
Received 980 / 480bit Protocol: 476
Received 984 / 480bit Protocol: 480
entering with changecount:73
472 984 10100001001000100010010111100110001
Received 152121137 / 24bit Protocol: 5
*********************************************
27.9 45% ch1
Received 1948 / 484bit Protocol: 488
Received 980 / 484bit Protocol: 484
entering with changecount:73
496 976 10010111000000100010111111100101101
Received 940670765 / 24bit Protocol: 5


Rinse and repeat for about 30 times and we start seeing the big picture. There are a lot of losses, buffer overflows and multi-threading issues (e.g. protocol 480 with 480 bits) but I can see that the significant values repeat themselves during a packet burst.

Decoding

Time display the leading bits as well and reformat the output:
CH3*****************************************
       110100001 0010 1000 11110010 0100 29.6 36%
        10100001 0010 1000 11110010 0101 29.6 37%
       110100001 0010 1000 11110010 0110 29.6 38%
        10100001 0010 1000 11110010 0110
       110100001 0010 1000 11110010 0111 29.6 39%
        10100001 0010 1001 11110010 0111 29.7 38%
        10100001 0010 1001 11110010 0111 29.7 39%
        10100001 0010 1001 11110010 1010 29.7 42%
     10110100001 0010 0100 11110010 0100 29.2 36%
      1010100001 0000 1111 11110001 1101 27.1 29%
          100001 0000 1111 11110001 1101
      1110100001 0000 1111 11110011 0001 27.1 49%
CH2*****************************************
       110010001 0001 1001 11110011 0000 28.1 48%
               1 0001 1001 11110011 0000
0100100110010001 0001 1001 11110011 0000
        10010001 0001 0010 11110011 0001 27.4 49%
CH1*****************************************
      1110000001 0001 0111 11110010 1101 27.9 41%
0100101110000001 0001 0111 11110010 1101
     10110000001 0001 0110 11110010 1110 27.8 46%
After more formatting:
CH3:110 10 0001 0010 1000 1111 0010 0100 29.6 36%
        3  -----296------      ----36--- 
     10 10 0001 0010 1001 1111 0010 1010 29.7 42%
           -----297------      ----42---
CH2:110 01 0001 0001 1001 1111 0011 0000 28.1 48%
CH1:110 00 0001 0001 0111 1111 0010 1101 27.9 41%
This shows that the structure is:
  • 10 bits header (and checksum?)
  • 2 bits: channel
  • 12 bits: temperature multiplied by 10
  • 4 bits: 1111 - padding(?)
  • 8 bits: humidity
I haven't studied the header checksumming formula but it's not needed for my purposes.
After removing the serial debugging lines from the decoder and plugging some quick&dirty code in the main loop...
if ( mySwitch.getReceivedProtocol() == 5){
  unsigned long value = mySwitch.getReceivedValue();
  unsigned int humidity = ((unsigned long)value & 0xFF);
  unsigned int temp = (((unsigned long)value >> 12) & 0xFFF);
  unsigned int channel = (((unsigned long)value >> 24) & 3) + 1;
  Serial.print( "Temp:");
  Serial.print( temp/10 );
  Serial.print( "." );
  Serial.print( temp%10 );
  Serial.print( "* Humidity:");
  Serial.print( humidity );
  Serial.print( "% Channel:");
  Serial.println( channel );
}
...the output is:

I spent a few hours into this, mostly understanding the rc-switch library and flashing the fake Arduino Nano with a fake PL2303HX serial converter. I've never seen so many blue screens in one day - until now.
As a bonus I've also decoded some RF-controlled outlets bought from Lidl 10 years ago. They were using the Quigg protocol*. Both these decoders will be opensourced once the code is cleaned up a bit.

If I were to do it again I would use the Stellaris Launchpad - it has step-by-step debugging and hands-free flashing.

*Actually, I was able to decode the RF outlets with the rc-switch library and the Stellaris, around one year ago, but somehow I could not do that with the Arduino Nano at the present time.

Wednesday, June 10, 2015

Essential Windows software for power users

Do you feel efficient enough when using Windows?
Here are some pieces of software that I use and have saved me years.

Total Commander

I've been using this for more than 10 years, think of it as Midnight Commander (mc) on steroids, for Windows. Screenshots of main features here: http://www.ghisler.com/screenshots/en/
It's practically the first thing I install when I need to work on a machine.

Scenarios:

  • copy stuff around from one drive/ftp/share to another, queuing (F2) operations along the way
  • view any files (F3) in ascii/unicode/hex, edit them (F4)
  • search in current folder (Alt+F7), much faster than using an IDE for source files
  • archive old files (Alt+F5, Alt+o) and unpack multiple archives (insert, Alt+F9), in background
  • compare directories (Shift+F2), including by content (Alt+c, y) like different GIT repositories
  • diff selected files, including binary/hex (Alt+f, y)
  • virtual folder support for FTP/Samba/nfs shares, adb, ext2fs, deleted files ...
  • virtual folder support for most archive types - I use mostly use zip with maximum compression
  • see how much each folder takes up (Alt+Shift+Enter)
  • open a quick DOS prompt at the current location (Alt+c, d) with the same elevation
There's an active community for add-ons and it's mature enough to be almost bug-free.
License: Winzip-like (forever shareware), 40 CHF (plus VAT!) otherwise.

IrfanView

Including the plugin pack this is my go-to image viewer and basic editor. It integrates nicely with the above (F3 on an image / video file to view it).

Scenarios:
  • open up a picture, press T to go to the thumbnails view and batch-organize files
  • select an area, press Ctrl+Y to crop to that area, press Ctrl+R to resize [the long side] to a size of your choice (I choose mostly 2048 px for the blog), Ctrl+S to save, space/PgDn to go to the next picture, DEL to remove the file
  • Shift+G to do basic gamma/contrast enhancement
  • L,R to rotate 90 deg, Ctrl+U to rotate arbitrarily (misaligned pictures)
  • B to run a batch conversion: change file names, resolution, format, EXIF
There's a lot of stuff that can be done, including support for old-style Photoshop plugins.

SysInternals

This one almost does not need an introduction. If it does, I suggest you take a look at 'A case of the unexplained' video presentations and blog posts.
I use Process Monitor a lot, especially to figure out why installers fail and what secret files some programs might access. For example a firmware update utility might write the raw firmware file to some other folder than Temp. Other programs check for a registry key + a hidden file to see when the trial expires.

PowerShell, WinDBG, BootICE, Unlocker and other utilities also fall under the same category, but they are too generic/specific/slow to use in comparison.

mobaXterm

I discovered this quite recently but it has become almost indispensable. It is like putty on steroids.
It allows managing several terminal sessions (ssh, telnet, ftp, serial!) at once, does X forwarding and saves the passwords as well. So you can have sessions for your router, Raspberry PIs, esp8266 and Arduino, all under one tabbed interface.

Also has SCP support out-of-the-box so you can browse the folders, view the files and edit them. I'm so happy whenever I don't have to use the vi retardness, especially on Solaris.

Best thing: it ports an almost complete Cygwin environment to Windows so you can run your awk/grep/cat/wc inside your DOS prompt. Close the window and it goes away.

License: free edition a bit lightweight (though sufficient), 49 EUR (+VAT) otherwise.

TightVNC

I use this to connect from my tablet or phone to the computer(s) running in the other room. Why get up from the bed if you forgot to shutdown the computer? Or if you want to stream some PC-only web stuff to Chromecast/EZCast in the other room, without dragging your laptop.

Notepad++

Another late-comer for me, I used to use RealEdit before.
I like the fact that it opens the same set of files so you can have a TODO list inside your Dropbox folder synced across computers. Unless it's a corporate workstation, that is.
But the editor in itself is really nice: lightweight, able to work with huge files, macros, syntax highlighting for a lot of languages, 
I will try to set it up for use with Arduino and light PHP work, it should support deployment scripts and autocompletion.

Classic Shell

I used to use it when Windows 8 first came out but decided to give the new Windows 10 launcher a chance. After 6 months of usage I can confidently say that Classic Shell is better. Text search works across the desktop icons as well, you can get rid of the heavy Metro sidebar and you can even disable Cortana. It's also noticeably faster in everything it does.

Honorable mentions

No offense to the Firebug team but Chrome dev tools really kick ass - live Javascript editing with two-way syncing with the workspace folder, live CSS editing, breakpoints that work 99% of the time.
Remove those 'click to like' overlays, debug why clicks are not working (hint: AdBlock).
I wish there would be one tool that does everything archive-related. TotalCmd really makes these almost obsolete. Still, there are times when they are needed.
For when someone loses files on their portable harddrive or formats the sd card from vacation.
A portable Apache / mongoDB / mySQL / PHP installation. The 'zero-installation' does not work on corporate workstations but you can make a batch file that maps a drive to a folder.
The swiss-knife of serial port debugging
You might be in for a big surprise after you calibrate your displays.
Deluge - non-classic mode
Use a Raspberry PI (or Banana PI or NAS) as your main torrent client, but manage it from any machine. The interface is similar to uTorrent, a bit buggy but very functional.
Make multi-boot USB sticks: Linux, Windows XP and Hiren - all on one stick
Plug any device into your Ethernet port and sniff the traffic.
The first one enables different settings (for example speed) on different pointing devices while the second one allows you to map your mouse buttons to do something else.
AutoHotKeyRecently started using this but there is a huge community of scripts. I like the fact that some Windows hotkeys can be disabled selectively.
Dexpot
Much better than SysInternals' Desktops.

Classic Start Menu, Snipping Tool, Task Manager, Resource Monitor (all three included in Windows), MPC-HC and VLC

Monday, June 8, 2015

Various laptop operations

I had these pictures lying around, they might be of help to someone. So, in no particular order:


  • MSI GT60 2PC mini-review, inside photo and getting the Windows 8 key from BIOS
  • Intel Centrino Advanced-N 6230 SMA socket repair
  • Random Toshiba laptop reflow attempt

Saturday, June 6, 2015

Keyboard membrane repair

This is a short article on a lighter note.
I got a broken keyboard (from boxes with DOA stuff) with a few keys that were not working.

I've opened the keyboard up and marked the keys that were not working, you can see a slight red marker trace in the picture below:


The key circuitry is a sandwich of three flexible plastic layers: top and bottom with conductive traces and a middle one for isolation. Only the bottom one was broken.

Using a continuity meter the breakage was traced to an area near the connector.
As a heads-up, the measured resistance for this type of traces is 5-10 ohms/cm, so for a strip/trace longer than 20 cm the meter might not beep anymore.

I had some silver paste sitting around unused, so I covered the broken traces with that.
 Some shaking required....


...after shaking all the silver flakes/powder should be evenly distributed. I don't think they mention this anywhere.



I believe a bottle of that stuff is around 15E for 5ml, but should last a really long time.

There's no way to properly construct traces, I've tried using toothpicks and other small-pointed objects:


I've left it to dry for 5 minutes then proceeded to remove the large unneeded areas with a cotton ball tip and traced the outline with a needle-like tip:


After 30 mins of drying and some more cleaning the keyboard was ready for use.

Reballing adventures - part 2

Thinking that I've had the process in hand, I've tried reballing a PS3 video chip. I'm not sure it was the video chip at fault since the person who gave it to me had attempted a reflow on it, transforming the YLOD (yellow light of death) into a GLOD (green one).

Setup was pretty similar to the one in the previous post; started with a light warming (100-120C) of the complete board to provide uniform thermal expansion and remove moisture:


However the chip proved to be a beast and my hot air gun was raising the temperature too slowly to be safe.

 I began looking for items that are a similar size and shape in order to make a template for the gun. This screwdriver accessory proved to be the best match.


The layout was started by wrapping some gardening wire around the magnetizer and holding it in place with some masking tape.

Not removing the tape proved to be a mistake when temperatures of 400C were involved.
The wired frame was then wrapped in food-grade aluminium foil...
...and after preheating the board I started following the lead-free solder temperature profile.
The attachment had to be moved really close to the board in order to get the temperatures to rise up fast enough: around 3 minutes are required.
Any longer than that and the chip can become toast, as well as the parts around it.

I tried to get the chip to be at 240C but it would not move. I've then removed the heat spreader (top aluminium part) exposing the bare chip and tried to remove that.

I was working in 3-5 minutes increments then slowly removing heat as to not stress the parts, then started a new cycle with 10C more.
After around 5 cycles of the above I raised the hot air temperature from 350C to 420C and see where that might go.

Still no luck, tried various tools to pry the chip off while still hot, thinking it might be attached with some strong glue.
Found out the hard way that the required temperature for removal was much higher, measured around 265C until solder balls started popping off and 280C until the chip could be removed.
Oh, I had to use a paint stripper for this, the hot air gun just could not provide sufficient thermal flow.

The result is ghastly:


The chip was permanently bent by the tools used to try to pry it, pads were ripped from the chip and pads and tracks were ripped from the motherboard.


I did not even attempt a reball at this point, there's no coming back to life for this unit.

Lessons learned:

  • have the proper attachment for the heat gun
  • ensure the gun can reach temperature in a safe time - or replace it if not
  • do not try to pry the chip - minor nudging should be enough to see if it's removable
  • most glues soften by the time the solder is molten
  • improper heating patterns (insufficient preheating) leads to delamination
  • same as above with regard to moisture content
  • eyeballing the temperature works much better in my case than using tools
  • prepare the full workflow in advance and work fast
I can see why reballing is such a lucrative (or rather expensive) service: the guys (and girls?) doing this have to break tens of units in order to teach themselves the proper process.

However, not everything is lost: I'm pretty confident now that I can prototype with BGAs with a 20% success rate. i.e.: 'Bad experience, F--, would attempt again'.

Reballing adventures - part 1

Some time ago I wrote about a reflow I did on a broken laptop:
http://hackcorrelation.blogspot.de/2014/01/medion-md97900-laptop-teardown-and.html

The unit worked fine for ~6 months, after which it failed again with the same symptom: no video.
It sat unused for one year but then I received a PS3 with the YLOD symptom. This prompted me to order a BGA reballing set from eBay and attempt to fix them both.

The PS3 reball will be in a part 2 of this series.
This is not a guide, just a journal describing the mistakes I've made during the process.

The video chip is the plagued NF-G6150 , also used in the HP DV6000. The laptop still has decent performance when paired with an SSD, for today's standards.


The area around the chip was masked with Kapton tape:


Then 'shielded' with food-grade aluminium:


I had a goose-neck-style phone cradle sitting around which provided a stable hands-free holder for the hot-air gun.


The chip was painted with a black marker (for reducing thermal reflection) and the measurement from an IR thermometer was compared to a K-type probe.


The hot air gun was left at 100C for around 15 minutes in order to heat the board uniformly to avoid a sudden thermal expansion on the working area.


After which the temperature was raised to ~350C on the gun (~250C on the chip), air speed was set to 3-4 (out of 7). This enabled a clean removal of the chip, but you can already see the non-uniformity on the surface pointing to a delamination.


The chip and motherboard were cleaned from solder - there are hundreds of how-to videos on YouTube.
The basic process: flux is added then the soldering iron is run with some fresh solder on the pads. After most of the solder has been removed, some more flux is added and the remaining solder is removed with a [copper] wire braid. The flux residue is cleaned with alcohol.





Did I mention how much I hate the Blogger editing interface? There's no way to align multiple pictures into the same area.

The chip is then covered in flux and the reballing pattern is placed over it.


A nice trick for getting the balls in (hehe) is to pour from the ball container into a larger container, over the pattern. Instead of the ~200 balls you will be using ~3k, but they can safely be reused afterwards. Learned this trick on YouTube from some random Indian guy:


The pattern and chip are then heated in order to reflow the balls and attach them to the chip


First attempt: 20% of the balls got detached from the chip and stuck on the template. Do not reuse these ones!


Second attempt: only two balls detached. They were manually placed on the pads and reflowed again, without the template in place.


The result looks promising:


Time to reflow the chip back into place. The black stuff is permanent marker that I've used to increase emissivity(?) or reduce heat reflection.


Result: no video.

I've tried two or three times to reflow the chip, I think I might've also reballed it again but there was no improvement. This one goes back to the scrap pile.

Inside stuff - Tenvis IPROBOT3 - WiFi IP camera

I got one of these units at a "gadget exchange meeting". The kind where both parties have mostly useless gadgets and they exchange them. Oh, the camera kept resetting as well.

Time to void some warranties.

Repairing a wireless Wiimote charger

Out of my eBay 'lucky bags' I got a wireless charger for Wii remotes. I forgot about it for more than one year since I did not have a Wii at that time.
It came with two battery placeholders that also replace the battery cover of the Wii remote. I've tested the included batteries and they seemed to still hold charge.


The unit would not turn on: it's time to void the warranty on this "Kamikaze Gear Wireless Charging Dock (Wii)":


Inside stuff - Auraton 2020 TX/RX thermostat

I got this pair of items from a colleague, requesting a repair. The units would not bind to each other, or rather, the RX unit (controlling the heating) would not receive anything from the TX unit.

The transmitter, pictured below, is quite a nice low-power unit. It contains the temperature sensor and provides a basic programming interface for controlling the heating - based on day of week, hour, vacation mode.


The receiver unit plugs into a free outlet, without hogging it:



I would like to see more products that take a plug space return it to the user. In this case, this plug would probably take 2 or even 3 spaces on a 45 degree German-style plug.

The manual for the unit is hosted here: http://www.auraton.pl/img/auraton/instrukcje/en/auraton_2020_2020txplus_en.pdf which also provided the "fix" for the radio binding problem.