Author: rb0unis

It’s show day and we need to get the car out of the lab, down the corridor, out of the building, into the next and get it set up to show off along with all the other demos.

The car has been on axle stands for the last 2 months so we lowered it back into the floor and drove it out of the lab.

Then we put it on the skids…

and pushed it back down the corridor….

drove it out of the building, did a couple of turns for fun around the Alan Turin statue and into the management school building where the celebration was…

once the car was over we took the Baxter demo over…

and the Sign language demo…

The event was a great success, see the celebration video here

https://youtu.be/Kk6o6H7a2K0

Then everything was packed up and taken back to CVSSP ready for the more informal Friday event back in the labs.

On Friday 5th everything was set up again in the lab…

and after another long but successful day, we celebrated with a BBQ and some drinks. Here are some group shots of the cognitive vision lab after all their hard work

I might be a little quite on the blog front for a while as I’m going to take a rest from the car. It’s taken some of every weekend for the last 2 months and a fair few evenings as well. Time to spend some time with the family. But we’ll get into phase 2 as quickly as possible.

I’m going to have to come back to the batter and power distribution. The car needs to be ready to show at the CVSSP 30th anniversary event on the 4th/5th April so we will demo the sensors using mains power.

With one day to go it’s time to tidy up, get the panels back on, give it a good clean and brand it.

We only needed to remove the wing mirrors, the panels over the door arch and the ones at the back of the door opening (so far) so that’s what needs to go back on. The first to go on is the one over the door, it just clips back in place using the original clips and a single screw. I needed to cut a small notch out of it to allow the metal band holding the roof bar clean passage. Did it by eye with a Dremel. 5 minute job. The rear panel is next and finally the wing mirrors which help hold the panels in place.

Then it was a quick clean and a wipe down with degreaser before the the decals could be applied. I’d been so wrapped up with teaching a new course in robotics which was consuming much of the week and the car at weekends, it was only when Helen “sorted me” that they got ordered. Unfortunately the lead time on dcals was too long but vinyl stickers were quick as was simple letters in fixed fonts.

We ordered “Surrey Autonomous Testbed” and “Deep Autonomous VEhicle” (Dave, the cars name) as vinyl decals. The 3 logos (AVP, CVSSP and Surrey Uni) were ordered as vinyl stickers and I’ll replace them later with dcals.

I won’t go into how to apply vinyl, there’s plenty of YouTube instructional videos for that (which is what I used). It’s fairly easy although the curves in the car are a challenge and I’m not convinced I chose the best lines to follow but hey ho….

Doesn’t look too bad.

Ok so the PX2 is in and connected to the sensors. Let’s boot it up and test if all the sensors work.

Flashing beacon light… check

Fans… check

Thermometer… check

and round the back

Atmospheric boot lighting… check

and the PX2 is alive…

This is the monitor in the boot, it’s showing the 6 camera feeds along the top. On the right are two SAND feature maps computed by a neural network on the fly using one of the GPU’s in the PX2. Jamie’s SAND features are from our 2019 CVPR publication:

http://personal.ee.surrey.ac.uk/Personal/R.Bowden/publications/2019/SpencerMartin_CVPR2019pp.pdf

In this image your can also see the lidar point cloud and the model of the Twizy, Oscar has been getting the software ready in prep.

It all works, although we’ve got some calibration to do now and there are more sensors that need to be added before phase 1 is complete such as the wheel odometer, parking sensors and GPS/IMU.

As more and more wires stacked up, I did start to wonder where the hell I’d put them all.,,.

We’ve already got the umbilical from the roof carrying 8 wires (6 camera, lidar and the power lead for the beacon). All but the latter needs to connect to the px2 and I can’t install that until the other wiring is in place.

The first thing we did was to wire USB, CAT6 and a HDMI through to the front of the cabin. I’d originally planned a HUD up front, perhaps using the 2nd HDMI on the px2, but I’ve also got a touch screen+ raspberry pi all in one compact case which might make a suitable alternative, we’ll see. Either way, having USB, network and HDMI means hopefully I don’t need to remove panels again in the future.

The wires come out of a cable gland near the bottom of the glove compartment. To remove this you need to remove the plastic pins holding the lid on, then undo the 4 torque screws at the top. There is another torque bolt, underneath the dash that also needs removing before you can pull out the plastic insert. We threaded the cables in the arch between the interior trim and the subframe

While I was at it, I threw in a power cable. The cables come into the boot on the right near the door catch secured in a cable gland and cable-tied to the frame to keep the pressure off the gland (the plastic panel of the wall is fairly flimsy.

All these cables were 4m so the excess at the front was coiled in the glove compartment. At the rear, they are coiled in the spare space at the bottom of the boot.

Next lights. It’s quite dark in the boot (I used a spotlight for much of the work so far). Rather than buy strip LED and have to wire them up I bought some really cheap ‘pimp my ride’ style rgb music activated lights.

I wasn’t interested in the sound activation but more the fact that for £10 i got 4 prewired led bars and a controller that was designed to work with a 12v car battery so should contain its own regulator. It also has a remote that allows the colour to be changed. White when working, blue for showing off. There’s that blue theme again.

Once uncoiled quite a few wires but with the help of Cihan and his magical Velcro cable ties (I’m a convert) we put one bar above the top shelf, one in the top shelf to light the px2 and two down at the battery level. In all cases, the leds where hidden behind framework as was 95% of the cable. The led lights were wired to a switch on the side panel, I then wired up the fans to one of the other switches and the beacon line from the roof umbilical to a 3rd.

I need to install the power distribution in the bottom along with battery and the charger. To do this I need to run a line back from the cars 12v battery back to the split charger for the leisure battery. The main drive battery for the Twizy is under the seat. I don’t want to touch this as it belongs to Renault but there is a small 12v lead acid battery behind the registration plate at the front. It’s used for the standard electrics on the vehicle such as lights and windscreen wiper and is kept topped up from the main lipo drive battery. I need to take a positive feed off this 12v battery back to an intelligent split charge relay so the leisure batteries in the boot can be recharged automatically from the Twizy. When we were installing the cables into the front cabin we spent a long time trying to also route a power cable down to the front battery but although we finally managed to route a cable down through behind the dash, I wasn’t happy about its proximity to the steering rack. After an hour of fighting to get a cable through, I abandoned this idea and decided to go under the vehicle later. We needed to demo in a few days so I decided we could demo on mains and I’d install the DC power circuitry later. It’s better that this power line is separate from the signal lines anyway.

However, to reduce structural work once he px2 was installed, I made some polycarbonate sheets that would hold the power distribution and fuses. On the small front panel I mounted two volt meters and an amp meter, to be wired later. This would show the main incoming voltage from the car battery which in turn is topped up by the Lipo bank of the car. The second one would show the leisure battery voltage and the current meter would show how much current the axillary electronics are using. Yet more parts from frank the robot. Poor frank.

Now it was finally time to mount the PX2. Using the tray I’d built previously, the px2 was installed on the first shelf, the cameras were routed down and the spare cable coiled neatly and hung from the roof of the boot to keep it out of the way. The px2 wiring loom was routed down toward the bottom. There is actually only one cable needed from this, the power, but I wasn’t prepared to rewrite the PX2 umbilical so they can also rest in the space at the bottom. We then connected everything to the mains and booted her up in situ for the first time.

Shes’s alive.

Obviously we need a monitor for working on the px2. This is not intended for use during driving, that was the purpose of the wires we routed to the front cabin. The monitor in the boot will be for debug. While one solution would be to plug in a monitor when we need it, I though it would be more convenient to have a small permantly installed monitor in the boot.

I found a 13.3 inch hd monitor on Amazon, super lightweight, metal cased with vesa mounts and importantly 12v. It has the added advantage that is could also be powered by a 2amp 5v USB feed. So I ordered this along with 2 different vesa mounts as I wasn’t sure which one would be best. Ironically when it arrived, the metal VESA mounting bracket is actually heavier than the monitor.

As with much of the car, the boot door in the cargo Twizy is largely plastic. The hinge is metal and upon investigation there is a horizontal metal bar that comes off the hindge and gives support for the door. So this metal bar was the best thing to mount to. Provided the monitor is not too heavy, it shouldn’t put excessive strain on the hinge and when the door is closed it’s obviously supported in both sides, the hinge on one side and the latch on the other. To gain access, I removed the rear light unit which allowed me to confirm the bar was metal. This is easily done by undoing the two bolts that hold the lighting unit to the metal cross bar from the inside.

The bulbs just unclip in their holders and dangle down by their wires. I drilled straight through the inside plastic and through the metal bar.

The vesa mount would have been too large for the bar if I used the original mounting holes so I used a bit of flat aluminium bar to reduce the distance between the mounting points.

and then mounted the monitor to the bracket with some thread locker for good measure, hey presto.

Time to start on the wiring. The roof has 6 cameras, a lidar and a beacon so that’s 8 wires. These need to be fed our through a cable gland and then braided into a single umbilical before breaking out to each device on the roof rack.

It was a bit fiddly as the largest gland I had wasn’t really big enough but after enlarging the hole they just about fitted with a bit of encouragement.

We staggered all the wires to reduce loose wire in the roof and cable tied everything to the framework.

The umbilical exits via a hole in the cut out above the seat belt and is then cable tied to the roof rack bar. It could probably have been tidier but it’ll do. Wired up the beacon and gave it a test.

Oscar checked all the cameras worked, although we forgot to check the lidar (fingers crossed).

There quite a bit of spare wire in the boot as it’s such a small car. Although it would be fairly easy to shorten the lidar cable, it’s difficult to shorten the camera leads. As these are the bulk of the cables, we will just coil the excess neatly somewhere.

The lidar and camera needs to go on the roof and the PX2, batteries etc need to go in the boot, so I need to build some sort of framework. Ideally this needs to attach to the subframe as it should be as rigid as possible. All the cameras and lidar are going to need calibrating and any movement will make that calibration useless. I ordered some extruded aluminium profile, bolts, corner brackets etc etc from a company called Motidis.

I think Motedis are based in Germany. The aluminium profile is surprisingly cheap, but I guess the money is made on the other items, as it quickly adds up. I could have got the stuff from RS or another UK site but I find the RS website a nightmare to navigate. I particularly liked the Motedis website as it listed all the accessories that matched each type of profile, so this made the choice a lot easier. The quantities were a conservative estimate of what I would need. This was based on a back of the envelope calculation that would provide some form of roof rack and a frame for the boot to mount the electronics etc. In the end I had enough alu profile but needed more corner brackets, short bolts, channel nuts, compression washers and nylon lock nuts. One thing to note, originally I ordered fixings with the brackets included but these turned out to be 4mm and I didn’t use them as there was just too much play in things. Pretty much all the aluminium frame was bolted with 5mm. Note the 5x10mm bolts are actually too long for the depth of the channel unless you use a split/compression washer. You actually need the 5x8mm. However, as I’d bought 10mm, I went with split washers but as these are notoriously useless at resisting vibration, every bolt had blue thredlocker applied. I had some rough sketches but what I wanted to achieve largely existed in my head and has evolved as the build has proceeded. However, once I took the panels off to get access to the subframe, it became difficult to see how the hell I would attach anything to the roof without serious modification to body panels. In the end, after a lot of scratching of heads and even trying to make some aluminium brackets which were going to make the car look more like an elf, I decided that maybe I could modify a roof rack without too much damage to the car. Very early on Id looked at roof racks and found nothing other than this video and a couple of blogs about dangerously clamping a Halfords roof rack to the plastic bodywork, but this would have been a bad idea as any flex in the plastic body panels would result in movement of the rood rack. However, despite having previously ruled out roof racks, late one night while desperately searching for a Twizy roof rack for the millionth time, I decided to buy a cheap universal roof rack from Amazon. As it clamped onto the roof via metal straps, I thought perhaps I could adapt it. Although the roof is semi rigid plastic and does flex under pressure, the edges where it would be clamped partially lie on the subframe, so this might work.

On Sat 9th Feb ’19 I went into work on a Sat morning, it was going to be the first of many Sat and Sundays in the lab. The first thing was to cut the roof rack to size. This involved drilling out the rivits, cutting the main bar to size then re-reviting it back together. 

T To mount to the subframe I needed to cut through the fibre glass cover that encases the subframe along the roof line. I did this with a dremmel, and it was dusty as hell. At the time I thought it was plastic, but I subsequently read somewhere its fibre glass, so I probably should have had a mask on !!! The cuts should be fine as they wont be visibile under the bodywork.

Although I found one place where a roof bar could be mounted, it wasn’t obvious where I could put a second due to the size and structure of the roof. So I decided the rear of the roof rack would be mounted on steel stud work that I could bring up through the roof and attach to the subframe internally in the boot. This would have the added advantage that I could use it to level and or change the angle of the roofrack.  As you can see, once the roof bar was moutned, the straps would still get in the way of the side panels over the doors. I spent a long time thinking how to get around this and finaly came to the conclusion that I shouldn’t worry about it,  we would probably never sell the car. Once we had finished with it, it would probably be scrapped and at some point I was going to have to start cutting holes in it. So I might as well get on with it. Let’s face it, who is going to want a used Twizy with no doors and full of hole so?? With the one roof bar and two 8mm studs protruding through waterproof glands in holes I cut in the back of the roof (see boot framework when I write it) this gave a pretty solid base on which to build some sort of custom roof rack. The basic frame was cut and bolted together from the extruded aluminium profile. Quite a few of the end connectors were tapped and bolted through to give extra strength. The remainder of the connections used 45′ angle brackets that Id got from Motedis but also galvanised steel corner brackets that I picked up at B&Q. The front bar was most of the width of the car, this is because I new we would need to mount the side cameras so they could see down over the edges to some extent. However, although we also needed a camera near the front, I didn’t extend the frame in that direction as I was concerned that doing so would result in a very unstable cantilevered structure which would bounce around making any calibration impossible. Id worry about it later, if the cameras were too far back on the structure, then they would need to be fixed directly to the roof above the windscreen. I did however, need to mount the lidar on the structure. The lidar has a range of +-15.5. The centre of the car where the lidar would be mounted was about 50cm from the back which means I would need to lift the Lidar up give it a clear view over the back of the car. Some simple trigonometry said the lidar needed raising so it had a clear view of the road and not the car/roof rack. I spent quite some time trying to mount underneath the roof rack bar to keep the whole thing as low as possible but there just wasn’t enough room space.

I got some 125x125x5mm alu offcuts cheap off eBay. Ideal size as slightly larger than the lidar, so I built the tower frame to fit which saved having to cut it. I bolted the lidar down with 8mm stud so not only does that hold the lidar down but compresses the whole tower to keep it rigid. 

At the weekend, once I’d mounted the cameras, I set to work cutting some holes in the bodywork through to the boot. Behind the drivers seat, the bulkhead that is between the main cabin and the boot is asymmetrical due to the fact that the near side is recessed at the top to give clearance for the seat belt. On the offside, you have a relatively clean and flat panel on which things can be mounted. Of course, this panel is potentially open to the elements, but I decided it would be a good place to mount some basic controls. Nothing too major, just switches to control basic electronics for lighting such as the flashing warning light, fans and bulkhead lights so you can actually see in the boot.  For most of the boot and the bulkhead is just a twin wall semi rigid plastic which is not structural, it’s easily cut with a jig saw. So I cut some appropriately sized holes for some switch panels I bought off Amazon. They were preassembled on a mounting panel and marketed for marine use and therefore waterproof. Not sure how much this is true, but I have no reason not to believe the description, yet….

In the figure you can see me cutting the holes and the final mounted panels (below). I chose these because they were waterproof and as when the switches are active, they have inbuilt blue LEDs which lights up the switch. As you will see, there’s a bit of a colour scheme evolving. 

Then on Tuesday I decided to stay late after work and get the fans mounted into the bulkhead. I decided we needed some ventilation as we have leisure batteries in there. They are sealed but still vent, so some ventilation is a good idea, just in case they gas. More importantly, we’re going to have computers in there (a px2 plus more possibly) running discrete gpu’s in anger, and these are going to get hot, so the boot needs cooling. I’ve got a digital thermometer that also needs mounting at some point.

Cutting the holes into the bulkhead was easy enough and fairly neat, but to cover the jagged edges I used some brake line tube I’d bought previously for a different purpose. I cut it open with a Stanley knife to line the edge of the cut and create a neat finish.

Mounting the fans was a pain in the arse, mainly due to the space/access, the size of the bolts and my fingers and some slightly off drilling. It took two of us, Oscar helped with the first and then late at night Cihan, who was working late, got roped in.

The rubber tube made a huge difference to the appearance and we are very pleased with our efforts.

Again, blue lights because they are cool. Although Cihan did point out I’d gone for style over function and the fans I’d chosen weren’t very efficient (or even the right kind!!).

Still, they looked good and they did move air through the boot and I can always change them if they overheat.

The view from the boot is below. The bottom fan is intake with a filter, the top fan is exhaust. The filter is on the inside, which isn’t ideal as the dust will collect on the side against the fan blades, but it was better aesthetically. Even if I’d put it on the outside I’d need to remove the grill to clean it so I attached it on the inside using short bolts to one side only so it could be removed without having to remove the bolts that hold the fan and grill to the bodywork.

The next day I found some time to add the thermometer and monitor. I’ll talk about the monitor when we install the px2. The thermometer was a cheap ebay purchase designed to be put in a PC as panel mount. I cut another small hole in the bulkhead and hey presto.

It has an inbuilt thermometer and external thermometer, which will be mounted high in the boot, roughly level with the top extraction fan.

The Sekonix cameras were the ones suggested by nvidia for use with the PX2. They came as either 120′ or 60′ field of view. We ordered 4 x 120′ to provide 360 vision around the vehicle which should allow 15′ overlap between each camera and 2x 60′ for a stereo pair that would be front facing. The street drone that Parkopedia ordered for the AVP project uses the same cameras but in addition to the stereo pair has 6x 120′, 1 at the front, one at the rear and 2 on each side. This configuration allows the camera overlap to be increased, which is no bad thing as the lens distortion makes the image less useful as we move further away from the optical axis of the camera. However, I was on a tight budget so I settled for the 4+2 configuration shown in the figure. After all, we could always add cameras at a later date. The PX2 actually supports up to 12 cameras via GMSL.

The cameras themselves are 1928 x 1208 resolution and supposedly waterproof (IP69K) as are the connectors so that’s the one less thing to worry about. However, as they are primarily aimed at automotive they are small (26x26x20.3) and do not have standard mounting points. A hunt of the web suggested Sekonix had some mounting brackets for several mounting options so I contacted them and they sent me the 3D CAD models. I printed one in pla on the printer (Hepestos 2) we have in the lab and it seemed ok. As shown in the figure it’s a basic cup designed to bolt to the back of the camera using 2 of the housing bolts, underneath it has a standard camera mount or at least space where a nut could be melted into the pla. My initial thought was that pla would be too brittle, especially given the lack of suspension on the Twizy. So I investigated getting some machined in alu. It was cheaper to outsource this than do it in-house (which is sad) but it was still going to Cost $100 per mount and mounting it on a tripod mount also filled me with fear about subsequently trying to calibrate it. So I put the decision on hold until I’d at least got a better idea of what the roof structure for the lidar would look like.

Once the extruded aluminium and lidar were mounted on the roof it made sense to mount the cameras to that. Originally we didn’t think it was forward enough on the vehicle to mount the front facing cameras and so I resigned myself to probably having to mount them directly to the roof itself above the windscreen. But some tests with the cameras themselves suggested that as long as they were mounted high on the lidar tower we were only going to loose around .5 meter (at most) of floor coverage in front of the vehicle. These tests also suggested that the side and back cameras should have an inclination of 45′, the front camera a 0′ inclination and the two stereo cameras 9′ inclination. The only issue is mounting the stereo cameras high on the lidar tower means a narrow stereo baseline. Putting them nearer the side cameras would give a wide baseline but much lower mounting position which means less coverage immediately in front of the car. I decided to go with the former and hopefully push the baseline out slightly by mounting either side of the tower. It can always be changed later if needs be.

The Motedis website also provides CAD models for the extruded aluminium which helped in building the mounts.

I used 123D shape to add brackets to the Sekonix camera mount but I had some problems creating a single solid model so once finished they were remeshed in mesh mixer, making them solid before exporting to stl and then to gcode via cura.

Photos or the model, the final print and it’s placement on the car are below along with the stl files.

You can download  the .stl files from Thingiverse here. I had no problems printing at all, didn’t even need to recalibrate the printer. Then I mounted them to the car. Unfortunately, on only one of the models did I remembered to put mounting holes (the last one I did), the rest I just used a 5mm drill bit. I normally print at 20% infill but all of these were printed in solid pla (100% infill) to try and make them as strong as possible, as such drilling them was both easy and produced very clean holes.

Oh and I stuck an amber flashing light on, because I thought it would be cool 🙂