Explaining pointers to a Javascript developer

Question

I started to learn coding backwards: high level first. This has the obvious liability of missing some basic concepts that I should definitely know, and when I try to learn a low level language, it throws me.

I have tried many times to understand pointers, however the explanations rapidly go over my head, usually because all of the example code uses languages that use pointers, which I don't understand other things about, and then I spin.

I am the most (and very at that) fluent in Javascript.

How would you explain pointers to a sad Javascript developer like me? Could someone provide me a practical, real life example?

Maybe even showing how, if Javascript had pointers, you could do x, and a pointer is different than a raw variable because of y.

Answer 1

Here's an attempt at a self-contained answer from first principles.

Pointers are part of a type system that permit the implementation of reference semantics. Here's how. We suppose that our language has a type system, by which every variable is of a certain type. C is a good example, but many languages work like this. So we can have a bunch of variables:

int a = 10;
int b = 25;

Further, we assume that function arguments are always copied from the caller scope into the function scope. (This is also true for many real languages, though the details can quickly become subtle when the type system gets 'hidden' from the user (e.g. such as in Java)). So let's have a function:

void foo(int x, int y);

When calling foo(a, b), the variables a and b are copied into local variables x and y corresponding to the formal parameters, and those copies are visible within the function scope. Whatever the function does with x and y has no effect on the variables a and b at the call site. The entire function call is opaque to the caller.

Now let's move on to pointers. A language with pointers contains, for every object type T, a related type T *, which is the type "pointer to T". Values of type T * are produced by taking the address of an existing object of type T. So a language that has pointers also needs to have a way to produce pointers, which is "taking the address of something". The purpose of a pointer is to store the address of an object.

But that's only one half of the picture. The other half is what to do with the address of an object. The main reason for caring about the address of an object is to be able to refer to the object whose address is being stored. This object is obtained by a second operation, suitably called dereferencing, which when applied to a pointer produces the object which is being "pointed to". Importantly, we do not a copy of the object, but we get the actual object.

In C, the address-of operator is spelled &, and the dereference operator is spelled *.

int * p = &a;    // p stores the address of 'a'

*p = 12;         // now a == 12

The first operand of the final assignment, *p, is the object a itself. Both a and *p are the same object.

Now why is this useful? Because we can pass pointers to functions to allow functions to change things outside the function's own scope. Pointers allow for indirection, and thus for referencing. You can tell the function about "something else". Here's the standard example:

void swap(int * p, int * q)
{
    int tmp = *p;
    *p = *q;
    *q = tmp;
}

We can tell the function swap about our variables a and b by giving it the addresses of those variables:

swap(&a, &b);

In this way, we are using pointers to implement reference semantics for the function swap. The function gets to refer to variables elsewhere and can modify them.

The fundamental mechanism of reference semantics can thus be summarized thus:

• The caller takes the address of the object to be refered to:

   T a;
   mangle_me(&a);
  

• The callee takes a pointer parameter and dereferneces the pointer to access the refered value.

   void mangle_me(T * p)
   {
       // use *p
   }
  

Reference semantics are important for may aspects of programming, and many programming languages supply them in some way or another. For example, C++ adds native reference support to the language, largely removing the needs for pointers. Go uses explicit pointers, but offers some notational "convenience" by sometimes automagically dereferencing a pointer. Java and Python "hide" pointer-ness inside their type system, e.g. the type of a variable is in some sense a pointer to the type of the object. In some languages, some types like ints are naked value types, and others (like lists and dictionaries) are "hidden-pointer-included" reference types. Your milage may vary.

Answer 2

C++ rules are fairly simple and consistent. I actually find how Javascript handles object references and prototypes way more unintuitive.

Preface A: Why is Javascript A Bad Place To Start?

The first thing you need to fundamentally understand before you can tackle pointers is variables. You need to know what they are and how the computer keeps track of them.

Coming from a Javascript background you are used to every variable assigned to an object being a reference. That is, two variables can reference the same object. This is essentially pointers without any syntax to allow for more intricate use. You are also used to implicit copies of "basic" types like numbers. That is to say:

var a = MyObject;
var b = a;

Now if you change b you also change a. You would need to explicitly copy MyObject in order to have two variables pointing to different instances of it!

var a = 5;
var b = a;

Now if you change b, a is not actually changed. This is because assigning a to b when a is a simple type will copy it automatically for you. You cannot get the same behavior as objects with simple numbers and vise versa, so when you want two variables to refer to the same number you have to wrap it in an object. There is no explicit way to indicate how you want to handle references vs copies for primitive types.

You can see this inconsistent behavior with no variation on syntax (but an extreme variation on behavior) can make the relationship between variables and what they contain muddy. For this reason I highly suggest banishing this mental model for a moment as we continue on our journey to understand explicit pointers.

Preface B: YOLO: Variable Lifetime On The Stack

So, let's talk from here on out in C++ terms. C++ is one of the most explicit languages in terms of what a variable is vs a pointer. C++ is a good entry point because it is low level enough to talk in terms of memory and lifespan, but high level enough to understand things at a decent level of abstraction.

So, in C++ when you create any variable it exists in a certain scope. There are two ways to create a variable, on the stack, and on the heap.

The stack refers to the call stack of your application. Every brace pair pushes a new context onto the stack (and pops it when it runs out). When you create a local variable, it exists in that particular stack frame, when that stack frame is popped the variable is destroyed.

A simple example of scope:

#include <iostream>
#include <string>

struct ScopeTest{
    ScopeTest(std::string a_name):
        name(a_name){
        std::cout << "Create " << name << std::endl;
    }
    ~ScopeTest(){
        std::cout << "Destroy " << name << std::endl;
    }
    ScopeTest(ScopeTest &a_copied){
        std::cout << "Copy " << a_copied.name << std::endl;
        name = a_copied.name + "(copy)";
        a_copied.name += "(original)";
    }

    std::string name;
};

ScopeTest getVariable(){ //Stack frame push
    ScopeTest c("c"); //Create c
    return c; //Copy c + Destroy c(original)
}

int main(){
    ScopeTest a("a"); //Create a
    {
        ScopeTest b("b"); //Create b
        ScopeTest d = getVariable();
    } //Destroy c(copy) + Destroy b
} //Destroy a

Output:

Create a
Create b
Create c
Copy c
Destroy c(original)
Destroy c(copy)
Destroy b
Destroy a

This should illustrate explicitly how a variable ties its life to the stack, how it is copied around, and when it dies.

Preface C: YOLO Variable Lifetime on the Heap

So, that's interesting conceptually, but variables can also be allocated outside of the stack, this is called "heap" memory because it is largely structure-less. The issue with heap memory is that you don't really have automatic cleanup based on scope. So you need a way to tie it to some kind of "handle" to keep track of it.

I'll illustrate here:

{
    new ScopeTest("a"); //Create a
} //Whoa, we haven't destroyed it!  Now we are leaking memory!

So, clearly we can't just say "new X" without keeping track of it. The memory gets allocated, but doesn't tie itself to a lifespan so it lives forever (like a memory vampire!)

In Javascript you can just tie it to a variable and the object dies when the last reference to it dies. Later I'll talk about a more advanced topic in C++ which allows for that, but for now let's look at simple pointers.

In C++ when you allocate a variable with new, the best way to track it is to assign it to a pointer.

Preface D: Pointers and The Heap

As I suggested, we can track allocated memory on the heap with a pointer. Our previous leaky program can be fixed like so:

{
    ScopeTest *a = new ScopeTest("a"); //Create a
    delete a; //Destroy a
}

ScopeTest *a; creates a pointer, and assigning it to a new ScopeTest("a") gives us a handle we can actually use to clean up and refer to the variable which exists in heap memory. I know heap memory sounds kinda confusing, but it's basically a jumble of memory that you can point to and say "hey you, I want a variable with no lifespan, make one and let me point at it".

Any variable created with the new keyword must be followed by exactly 1 (and no more than 1) delete or it will live forever, using up memory. If you try to delete any memory address other than 0 (which is a no-op) more than one time, you could be deleting memory not under your program's control which results in undefined behavior.

ScopeTest *a; declares a pointer. From here on out, any time you say "a" you are referring to a specific memory address. *a will refer to the actual object at that memory address, and you can access properties of it (*a).name. a-> in C++ is a special operator that does the same thing as (*a).

{
    ScopeTest *a = new ScopeTest("a"); //Create a
    std::cout << a << ": " << (*a).name << ", " << a->name << std::endl;
    delete a; //Destroy a
}

Output for the above will look something like:

007FB430: a, a

Where 007FB430 is a hex representation of a memory address.

So in the purest sense, a pointer is literally a memory address and the ability to treat that address as a variable.

The Relationship Between Pointers and Variables

We don't just have to use pointers with heap allocated memory though! We can assign a pointer to any memory, even memory living on the stack. Just be careful your pointer doesn't out-live the memory it points to or you'll have a dangling pointer which could do bad things if you continue to try and use it.

It is always the programmer's job to make sure a pointer is valid, there are literally 0 checks in place in C++ to help you out when dealing with bare memory.

int a = 5; //variable named a has a value of 5.
int *pA = &a; //pointer named pA is now referencing the memory address of a (we reference "a" with & to get the address).

Now pA refers to the same value as &a, that is to say, it is the address of a.

*pA refers to the same value as a.

You can treat *pA = 6; the same as a = 6. Observe (continuing from the above two lines of code):

std::cout << *pA << ", " << a << std::endl; //output 5, 5
a = 6;
std::cout << *pA << ", " << a << std::endl; //output 6, 6
*pA = 7;
std::cout << *pA << ", " << a << std::endl; //output 7, 7

You can see why *pA is called a "pointer". It is literally pointing to the same address in memory as a. So far we have been using *pA to de-reference the pointer and access the value at the address it points to.

Pointers have a few interesting properties. One of those properties is that it can change the object it is pointing at.

int b = 20;
pA = &b;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 20, 7, 20
*pA = 25;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 25, 7, 25
pA = &a;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 7, 7, 25
*pA = 8;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 8, 8, 25
b = 30;
pA = &b;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 30, 8, 30

So you can see that a pointer is really just a handle to a point in memory. This can be exceptionally useful in many cases, do not write it off just because this sample is simplistic.

---

Now, the next thing you need to know about pointers is that you can increment them as long as the memory you are incrementing to belongs to your program. The most common example is C strings. In modern C++ strings are stored in a container called std::string, use that, but I will use an old C style string to demonstrate array access with a pointer.

Pay close attention to ++letter. What this does is increment the memory address the pointer is looking at by the size of the type it is pointing to.

Let's break this down a bit more, re-read the above sentence a few times then continue on.

If I have a type that is sizeof(T) == 4, every ++myPointerValue will shift 4 spaces in memory to point to the next "value" of that type. This is part of why the pointer "type" matters.

char text[] { 'H', 'e', 'l', 'l', 'o', '\0' }; //could be char text[] = "Hello"; but I want to show the \0 explicitly
char* letter = text;

for (char* letter = &text[0]; *letter != '\0';++letter){
    std::cout << "[" << *letter << "]";
}
std::cout << std::endl;

The above will loop over the string as long as there is no '\0' (null) character. Keep in mind this can be dangerous and is a common source of insecurity in programs. Assuming your array is terminated by some value, but then getting an array that overflows allowing you to read arbitrary memory. That's a high level description anyway.

For that reason it is much better to be explicit with string length and use safer methods such as std::string in regular use.

---

Alright, and as a final example to put things into context. Let's say I have several discreet "cells" that I want to link together into one coherent "list". The most natural implementation of this with non-contiguous memory is to use pointers to direct each node to the next one in the sequence.

With pointers you can create all sorts of complex data structures, trees, lists, and more!

struct Node {
    int value = 0;
    Node* previous = nullptr;
    Node* next = nullptr;
};
struct List {
    List(){
        head = new Node();
        tail = head;
    }
    ~List(){
        std::cout << "Destructor: " << std::endl;
        Node* current = head;
        while (current != nullptr){
            Node* next = current->next;
            std::cout << "Deleting: " << current->value << std::endl;
            delete current;
            current = next;
        }
    }
    void Append(int value){
        Node* previous = tail;
        tail = new Node();
        tail->value = value;
        tail->previous = previous;
        previous->next = tail;
    }

    void Print(){
        std::cout << "Printing the List:" << std::endl;
        Node* current = head;
        for (Node* current = head; current != nullptr;current = current->next){
            std::cout << current->value << std::endl;
        }
    }
    Node* tail;
    Node* head;
};

And putting it to use:

List sampleList;
sampleList.Append(5);
sampleList.Append(6);
sampleList.Append(7);
sampleList.Append(8);
sampleList.Print();

List may seem complicated at a glance, but I am not introducing any new concepts here. This is exactly the same things I covered above, just implemented with a purpose.

Homework for you to completely understand pointers would be to provide two methods in List:

1. Node* NodeForIndex(int index)
2. void InsertNodeAtIndex(int index, int value)

This list implementation is exceptionally poor. std::list is a much better example, but it most cases due to data locality you really want to stick with std::vector. Pointers are exceptionally powerful tools, and fundamental in computer science. You need to understand them to appreciate how the common data types you rely on every day are composed, and in time you will come to appreciate the explicit separation of value from pointer in C++.

Beyond simple pointers: std::shared_ptr

std::shared_ptr gives C++ the ability to deal with reference counted pointers. That is to say, it gives a similar behavior to Javascript object assignment (where an object is destroyed when the last reference to that object is set to null or destroyed).

std::shared_ptr is just like any other stack based variable. It ties its lifetime to the stack, and then holds a pointer to memory allocated on the heap. In this regard, it encapsulates the concept of a pointer in a safer manner than having to remember to delete.

Let's re-visit our earlier example that did leak memory:

{
    new ScopeTest("a"); //Create a
} //Whoa, we haven't destroyed it!  Now we are leaking memory!

With a shared_ptr we can do the following:

{
    std::shared_ptr<ScopeTest> a(new ScopeTest("a")); //Create a
}//Destroy a

And, a little more complex:

{
    std::shared_ptr<ScopeTest> showingSharedOwnership;
    {
        std::shared_ptr<ScopeTest> a(new ScopeTest("a")); //"Create a" (ref count 1)
        showingSharedOwnership = a; //increments a's ref count by 1. (now 2)
    } //the shared_ptr named a is destroyed, decrements ref count by 1. (now 1)
} //"Destroy a" showingSharedOwnership dies and decrements the ref count by 1. (now 0)

I won't go too much further here, but this should open your mind to pointers.